[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:51:39 dist-test-slave-6zbq krb5kdc[24436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:51:39 dist-test-slave-6zbq krb5kdc[24436](info): set up 2 sockets
Aug 09 18:51:39 dist-test-slave-6zbq krb5kdc[24436](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 18:51:40.704327 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:51:40 dist-test-slave-6zbq krb5kdc[24436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:51:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:51:40.807132 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:42825
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:42825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:40.883464 24458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:40.883639 24458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:40.883677 24458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:40.884760 24458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:51:40.884799 24458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:40.884817 24458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:40.884833 24458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:51:40.884850 24458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:51:40.886258 24458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:42825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:42825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:40.886497 24458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:40.886689 24458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:40.891377 24468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:40.891487 24471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:40.891428 24469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:40.891423 24458 server_base.cc:1034] running on GCE node
I20260809 18:51:40.891770 24458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:40.892035 24458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:40.893172 24458 hybrid_clock.cc:648] HybridClock initialized: now 1786301500893159 us; error 78 us; skew 500 ppm
Aug 09 18:51:40 dist-test-slave-6zbq krb5kdc[24436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301500, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:40.895331 24458 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:51:40.895735 24458 webserver.cc:460] Webserver started at http://127.23.218.254:37675/ using document root <none> and password file <none>
I20260809 18:51:40.895912 24458 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:40.895957 24458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:40.896075 24458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:40.897058 24458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "1550a7e6ccb34063bc6305ce42753907"
format_stamp: "Formatted at 2026-08-09 18:51:40 on dist-test-slave-6zbq"
server_key: "48726f08694195919b681e5e2ab8d5cf"
server_key_iv: "8b0e1d33c323b02481ee4a7bde1d9cda"
server_key_version: "encryptionkey@0"
I20260809 18:51:40.897378 24458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "1550a7e6ccb34063bc6305ce42753907"
format_stamp: "Formatted at 2026-08-09 18:51:40 on dist-test-slave-6zbq"
server_key: "48726f08694195919b681e5e2ab8d5cf"
server_key_iv: "8b0e1d33c323b02481ee4a7bde1d9cda"
server_key_version: "encryptionkey@0"
I20260809 18:51:40.898564 24458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:51:40.899185 24485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:40.899327 24458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:40.899405 24458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "1550a7e6ccb34063bc6305ce42753907"
format_stamp: "Formatted at 2026-08-09 18:51:40 on dist-test-slave-6zbq"
server_key: "48726f08694195919b681e5e2ab8d5cf"
server_key_iv: "8b0e1d33c323b02481ee4a7bde1d9cda"
server_key_version: "encryptionkey@0"
I20260809 18:51:40.899462 24458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:40.905617 24458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:40.907145 24458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:40.907262 24458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:40.910884 24458 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:42825
I20260809 18:51:40.910919 24572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:42825 every 8 connection(s)
I20260809 18:51:40.911193 24458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:51:40.911918 24574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:40.913882 24574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907: Bootstrap starting.
I20260809 18:51:40.914420 24574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:40.914654 24574 log.cc:824] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:40.915242 24574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907: No bootstrap required, opened a new log
I20260809 18:51:40.916684 24574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1550a7e6ccb34063bc6305ce42753907" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 42825 } }
I20260809 18:51:40.916806 24574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:40.916831 24574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1550a7e6ccb34063bc6305ce42753907, State: Initialized, Role: FOLLOWER
I20260809 18:51:40.916925 24574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1550a7e6ccb34063bc6305ce42753907" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 42825 } }
I20260809 18:51:40.916973 24574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:51:40.916991 24574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:51:40.917021 24574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:40.917452 24574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1550a7e6ccb34063bc6305ce42753907" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 42825 } }
I20260809 18:51:40.917534 24574 leader_election.cc:302] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1550a7e6ccb34063bc6305ce42753907; no voters:
I20260809 18:51:40.917644 24574 leader_election.cc:288] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:51:40.917688 24581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:40.917843 24581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [term 1 LEADER]: Becoming Leader. State: Replica: 1550a7e6ccb34063bc6305ce42753907, State: Running, Role: LEADER
I20260809 18:51:40.917847 24574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:51:40.917949 24581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1550a7e6ccb34063bc6305ce42753907" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 42825 } }
I20260809 18:51:40.918435 24583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1550a7e6ccb34063bc6305ce42753907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1550a7e6ccb34063bc6305ce42753907" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 42825 } } }
I20260809 18:51:40.918521 24583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:40.918943 24584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1550a7e6ccb34063bc6305ce42753907. Latest consensus state: current_term: 1 leader_uuid: "1550a7e6ccb34063bc6305ce42753907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1550a7e6ccb34063bc6305ce42753907" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 42825 } } }
I20260809 18:51:40.919030 24584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907 [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:40.919113 24593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:51:40.919554 24593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:51:40.920776 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 24458
I20260809 18:51:40.920861 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:51:40.921028 24593 catalog_manager.cc:1347] Generated new cluster ID: 82dd53c13cb54184b8d3b779b49ab958
I20260809 18:51:40.921062 24593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:51:40.921591 24427 external_mini_cluster.cc:1428] Setting key 62584522436bbfbbb14234740092ffe5
I20260809 18:51:40.926504 24593 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:51:40.927083 24593 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 18:51:40 dist-test-slave-6zbq krb5kdc[24436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:40.937680 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:40.922872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.23.218.254:42825)
0809 18:51:40.923029 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:40.923037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:40.923066 (+ 29us) server_negotiation.cc:407] Connection header received
0809 18:51:40.924896 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:40.924903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:40.924954 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:40.925090 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:40.926938 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:40.928300 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:40.929670 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:40.930282 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:40.930415 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:40.934139 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:40.934148 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:40.934149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:40.934160 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:40.935911 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:40.936371 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:40.936373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:40.936374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:40.936464 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:40.936708 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:40.936709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:40.936709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:40.936935 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:40.937074 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:40.937326 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 18:51:40.937456 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260809 18:51:40.938417 24593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1550a7e6ccb34063bc6305ce42753907: Generated new TSK 0
I20260809 18:51:40.938570 24593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:51:40.965175 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:42825
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:41.042935 24614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:41.043107 24614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:41.043128 24614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:41.044232 24614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:41.044273 24614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:41.044301 24614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:51:41.045702 24614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:42825
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:41.045910 24614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:41.046109 24614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:41.050625 24624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:41.050637 24622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:41.050738 24621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:41.050987 24614 server_base.cc:1034] running on GCE node
I20260809 18:51:41.051167 24614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:41.051390 24614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:41.052528 24614 hybrid_clock.cc:648] HybridClock initialized: now 1786301501052516 us; error 27 us; skew 500 ppm
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:41.054932 24614 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:51:41.055301 24614 webserver.cc:460] Webserver started at http://127.23.218.193:44309/ using document root <none> and password file <none>
I20260809 18:51:41.055488 24614 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:41.055526 24614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:41.055608 24614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:41.056499 24614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "7bcf17f6ea06464aaefe4c7459f7b14c"
format_stamp: "Formatted at 2026-08-09 18:51:41 on dist-test-slave-6zbq"
server_key: "eb3b521233f8e50403dafb7c45f915e7"
server_key_iv: "cb6a63f453dcbab05cdec7712e6c632b"
server_key_version: "encryptionkey@0"
I20260809 18:51:41.056768 24614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "7bcf17f6ea06464aaefe4c7459f7b14c"
format_stamp: "Formatted at 2026-08-09 18:51:41 on dist-test-slave-6zbq"
server_key: "eb3b521233f8e50403dafb7c45f915e7"
server_key_iv: "cb6a63f453dcbab05cdec7712e6c632b"
server_key_version: "encryptionkey@0"
I20260809 18:51:41.057868 24614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:51:41.058470 24638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.058614 24614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.058687 24614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "7bcf17f6ea06464aaefe4c7459f7b14c"
format_stamp: "Formatted at 2026-08-09 18:51:41 on dist-test-slave-6zbq"
server_key: "eb3b521233f8e50403dafb7c45f915e7"
server_key_iv: "cb6a63f453dcbab05cdec7712e6c632b"
server_key_version: "encryptionkey@0"
I20260809 18:51:41.058758 24614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:41.086604 24614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:41.088224 24614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:41.088352 24614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:41.088542 24614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:41.088819 24614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:41.088851 24614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.088877 24614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:41.088894 24614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.094364 24614 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:37537
I20260809 18:51:41.094413 24813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:37537 every 8 connection(s)
I20260809 18:51:41.094699 24614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:51:41.099157 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 24614
I20260809 18:51:41.099257 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:51:41.099412 24427 external_mini_cluster.cc:1428] Setting key c111783819d2cf2e29f0d1566fd33fcd
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:41.103503 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:41.095431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:37769 (local address 127.23.218.254:42825)
0809 18:51:41.095490 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:41.095492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:41.095670 (+ 178us) server_negotiation.cc:407] Connection header received
0809 18:51:41.096134 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:41.096135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:41.096154 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:41.096202 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:41.097059 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.097555 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.098475 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.098667 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.098716 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:41.100882 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:41.100883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:41.100883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:41.100890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:41.102152 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.102621 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.102622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.102622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.102678 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.102938 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.102939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.102939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.103033 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:41.103081 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:41.103367 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 18:51:41.103441 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 18:51:41.103689 24817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:41.095407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:42825 (local address 127.23.218.193:37769)
0809 18:51:41.095562 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:41.095596 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:41.096055 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:41.096256 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:41.096259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:41.096814 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:41.096993 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:41.096997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.097613 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.097614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:41.098416 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:41.098418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.098726 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.098726 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:41.098827 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:41.099247 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:41.099257 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:41.100785 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:41.102241 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:41.102248 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:41.102250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:41.102550 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:41.102740 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:41.102741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:41.102741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:41.102885 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:41.103086 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:41.103092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:41.103247 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 18:51:41.103346 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 18:51:41.103492 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260809 18:51:41.104236 24814 heartbeater.cc:347] Connected to a master server at 127.23.218.254:42825
I20260809 18:51:41.104334 24814 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:41.104513 24814 heartbeater.cc:511] Master 127.23.218.254:42825 requested a full tablet report, sending...
I20260809 18:51:41.105010 24514 ts_manager.cc:194] Registered new tserver with Master: 7bcf17f6ea06464aaefe4c7459f7b14c (127.23.218.193:37537)
I20260809 18:51:41.105578 24514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:37769
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:51:41.122164 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:42825
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:41.200521 24820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:41.200700 24820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:41.200723 24820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:41.201860 24820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:41.201900 24820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:41.201924 24820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:51:41.203390 24820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:42825
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:41.203588 24820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:41.203781 24820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:41.208279 24831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:41.208304 24826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:41.208371 24820 server_base.cc:1034] running on GCE node
W20260809 18:51:41.208278 24827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:41.208703 24820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:41.208914 24820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:41.210038 24820 hybrid_clock.cc:648] HybridClock initialized: now 1786301501210032 us; error 29 us; skew 500 ppm
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:41.212263 24820 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:51:41.212635 24820 webserver.cc:460] Webserver started at http://127.23.218.194:44701/ using document root <none> and password file <none>
I20260809 18:51:41.212822 24820 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:41.212867 24820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:41.212984 24820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:41.213896 24820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "bf8a416739914afd9d360e26cb7c7719"
format_stamp: "Formatted at 2026-08-09 18:51:41 on dist-test-slave-6zbq"
server_key: "638d473c136589b1ffa82e0f8af1a936"
server_key_iv: "8fc9aad0aa277856344542f65aaac0c0"
server_key_version: "encryptionkey@0"
I20260809 18:51:41.214203 24820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "bf8a416739914afd9d360e26cb7c7719"
format_stamp: "Formatted at 2026-08-09 18:51:41 on dist-test-slave-6zbq"
server_key: "638d473c136589b1ffa82e0f8af1a936"
server_key_iv: "8fc9aad0aa277856344542f65aaac0c0"
server_key_version: "encryptionkey@0"
I20260809 18:51:41.215408 24820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:51:41.216041 24840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.216183 24820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:41.216261 24820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "bf8a416739914afd9d360e26cb7c7719"
format_stamp: "Formatted at 2026-08-09 18:51:41 on dist-test-slave-6zbq"
server_key: "638d473c136589b1ffa82e0f8af1a936"
server_key_iv: "8fc9aad0aa277856344542f65aaac0c0"
server_key_version: "encryptionkey@0"
I20260809 18:51:41.216325 24820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:41.226011 24820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:41.227885 24820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:41.228008 24820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:41.228186 24820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:41.228431 24820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:41.228462 24820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.228488 24820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:41.228504 24820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.233760 24820 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:41017
I20260809 18:51:41.233815 25024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:41017 every 8 connection(s)
I20260809 18:51:41.234045 24820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:51:41.235801 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 24820
I20260809 18:51:41.235872 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:51:41.236004 24427 external_mini_cluster.cc:1428] Setting key 49a76d16394fa39bd5820425a0db831c
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:41.242508 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:41.234614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:47105 (local address 127.23.218.254:42825)
0809 18:51:41.234659 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:41.234661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:41.234857 (+ 196us) server_negotiation.cc:407] Connection header received
0809 18:51:41.235240 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:41.235240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:41.235249 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:41.235296 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:41.236135 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.236779 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.237679 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.237948 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.238016 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:41.240047 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:41.240048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:41.240049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:41.240056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:41.241290 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.241647 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.241648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.241648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.241698 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.241907 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.241908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.241908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.242010 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:41.242066 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:41.242314 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 18:51:41.242421 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:51:41.242676 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:41.234587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:42825 (local address 127.23.218.194:47105)
0809 18:51:41.234746 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:41.234781 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:41.235171 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:41.235348 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:41.235351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:41.235837 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:41.236050 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:41.236056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.236859 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.236860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:41.237596 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:41.237598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.238029 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.238030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:41.238172 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:41.238595 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:41.238605 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:41.239961 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:41.241366 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:41.241373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:41.241375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:41.241582 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:41.241752 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:41.241753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:41.241753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:41.241852 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:41.242065 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:41.242079 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:41.242200 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 18:51:41.242282 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 18:51:41.242442 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260809 18:51:41.243111 25026 heartbeater.cc:347] Connected to a master server at 127.23.218.254:42825
I20260809 18:51:41.243211 25026 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:41.243381 25026 heartbeater.cc:511] Master 127.23.218.254:42825 requested a full tablet report, sending...
I20260809 18:51:41.243736 24514 ts_manager.cc:194] Registered new tserver with Master: bf8a416739914afd9d360e26cb7c7719 (127.23.218.194:41017)
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
I20260809 18:51:41.244204 24514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:47105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:51:41.258026 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:42825
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:41.336366 25037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:41.336537 25037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:41.336560 25037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:41.337659 25037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:41.337698 25037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:41.337728 25037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:51:41.339166 25037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:42825
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:41.339355 25037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:41.339536 25037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:41.344049 25050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:41.344028 25048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:41.344036 25045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:41.344177 25037 server_base.cc:1034] running on GCE node
I20260809 18:51:41.344424 25037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:41.344601 25037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:41.345738 25037 hybrid_clock.cc:648] HybridClock initialized: now 1786301501345724 us; error 31 us; skew 500 ppm
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:41.347967 25037 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:51:41.348335 25037 webserver.cc:460] Webserver started at http://127.23.218.195:40871/ using document root <none> and password file <none>
I20260809 18:51:41.348510 25037 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:41.348554 25037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:41.348670 25037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:41.349643 25037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "3d886bc95b6347e4946997e31eb9b26e"
format_stamp: "Formatted at 2026-08-09 18:51:41 on dist-test-slave-6zbq"
server_key: "40ff65829400bfbe81c9d71bee2c345a"
server_key_iv: "d678b3ec201a9b12189bde9a4bf7327e"
server_key_version: "encryptionkey@0"
I20260809 18:51:41.350013 25037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "3d886bc95b6347e4946997e31eb9b26e"
format_stamp: "Formatted at 2026-08-09 18:51:41 on dist-test-slave-6zbq"
server_key: "40ff65829400bfbe81c9d71bee2c345a"
server_key_iv: "d678b3ec201a9b12189bde9a4bf7327e"
server_key_version: "encryptionkey@0"
I20260809 18:51:41.351263 25037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:51:41.351941 25061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.352093 25037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:41.352173 25037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "3d886bc95b6347e4946997e31eb9b26e"
format_stamp: "Formatted at 2026-08-09 18:51:41 on dist-test-slave-6zbq"
server_key: "40ff65829400bfbe81c9d71bee2c345a"
server_key_iv: "d678b3ec201a9b12189bde9a4bf7327e"
server_key_version: "encryptionkey@0"
I20260809 18:51:41.352234 25037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:41.374547 25037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:41.376433 25037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:41.376557 25037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:41.376756 25037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:41.377010 25037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:41.377041 25037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.377066 25037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:41.377084 25037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:41.382603 25037 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:34467
I20260809 18:51:41.382678 25247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:34467 every 8 connection(s)
I20260809 18:51:41.382907 25037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:41.390825 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:41.383626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:59483 (local address 127.23.218.254:42825)
0809 18:51:41.383672 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:41.383674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:41.383838 (+ 164us) server_negotiation.cc:407] Connection header received
0809 18:51:41.384252 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:41.384253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:41.384262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:41.384312 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:41.385170 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.385637 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.386575 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.386751 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.386795 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:41.388570 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:41.388571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:41.388571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:41.388578 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:41.389749 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.390069 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.390070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.390070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.390111 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.390386 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.390386 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.390386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.390474 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:41.390517 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:41.390713 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 18:51:41.390765 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:51:41.390980 25251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:41.383560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:42825 (local address 127.23.218.195:59483)
0809 18:51:41.383736 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:41.383772 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:41.384182 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:41.384371 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:41.384374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:41.384866 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:41.385037 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:41.385042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.385700 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.385701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:41.386514 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:41.386516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.386803 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.386804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:41.386890 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:41.387256 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:41.387263 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:41.388491 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:41.389817 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:41.389822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:41.389824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:41.390016 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:41.390195 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:41.390196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:41.390197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:41.390301 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:41.390517 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:41.390520 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:41.390619 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 18:51:41.390695 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 18:51:41.390806 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260809 18:51:41.391379 25248 heartbeater.cc:347] Connected to a master server at 127.23.218.254:42825
I20260809 18:51:41.391472 25248 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:41.391620 25248 heartbeater.cc:511] Master 127.23.218.254:42825 requested a full tablet report, sending...
I20260809 18:51:41.391944 24514 ts_manager.cc:194] Registered new tserver with Master: 3d886bc95b6347e4946997e31eb9b26e (127.23.218.195:34467)
I20260809 18:51:41.391976 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25037
I20260809 18:51:41.392050 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:51:41.392210 24427 external_mini_cluster.cc:1428] Setting key 6ad54fa8be2a9594abe3fd31c4061e70
I20260809 18:51:41.392355 24514 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:59483
I20260809 18:51:41.393225 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:51:41.414927 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:41.408326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40874 (local address 127.23.218.193:37537)
0809 18:51:41.408481 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:41.408487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:41.408495 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:51:41.408516 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:41.408517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:41.408530 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:41.408586 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:41.409102 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.409772 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.410474 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.410785 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.410843 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:41.412714 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:41.412721 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:41.412721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:41.412728 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:41.413894 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.414193 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.414196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.414196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.414253 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.414443 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.414444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.414444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.414565 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:41.414712 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:41.414758 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 18:51:41.414830 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
W20260809 18:51:41.415244 24680 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40874
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[24436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:51:41.434252 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:41.428796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40876 (local address 127.23.218.193:37537)
0809 18:51:41.428855 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:41.428857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:41.428903 (+ 46us) server_negotiation.cc:407] Connection header received
0809 18:51:41.428940 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:41.428941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:41.428950 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:41.428991 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:41.429424 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.429857 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.430486 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:41.430673 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:41.430717 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:41.432413 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:41.432414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:41.432414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:41.432420 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:41.433489 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.433771 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.433772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.433772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.433815 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:41.433937 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:41.433937 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:41.433938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:41.434009 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:41.434085 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:41.434142 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 18:51:41.434197 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 18:51:41.434700 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 24614
I20260809 18:51:41.449363 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 24820
I20260809 18:51:41.456259 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25037
I20260809 18:51:41.463718 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 24458
2026-08-09T18:51:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1930 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[25271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[25271](info): set up 2 sockets
Aug 09 18:51:41 dist-test-slave-6zbq krb5kdc[25271](info): commencing operation
krb5kdc: starting...
W20260809 18:51:42.591791 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:51:42 dist-test-slave-6zbq krb5kdc[25271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:51:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:51:42.698886 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34861
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:34861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:42.776333 25289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:42.776506 25289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:42.776535 25289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:42.777619 25289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:51:42.777658 25289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:42.777684 25289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:42.777702 25289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:51:42.777725 25289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:51:42.779191 25289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:34861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:42.779402 25289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:42.779589 25289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:42.783955 25297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:42.783967 25301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:42.783953 25298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:42.784181 25289 server_base.cc:1034] running on GCE node
I20260809 18:51:42.784439 25289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:42.784726 25289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:42.785869 25289 hybrid_clock.cc:648] HybridClock initialized: now 1786301502785855 us; error 30 us; skew 500 ppm
Aug 09 18:51:42 dist-test-slave-6zbq krb5kdc[25271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301502, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:42.791106 25289 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:51:42.791474 25289 webserver.cc:460] Webserver started at http://127.23.218.254:43283/ using document root <none> and password file <none>
I20260809 18:51:42.791685 25289 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:42.791733 25289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:42.791852 25289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:42.792820 25289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "20a5ae7efb1f42c7a9321043df56ea37"
format_stamp: "Formatted at 2026-08-09 18:51:42 on dist-test-slave-6zbq"
server_key: "828285449a1fa2f24ef8d7f6b3a65b4c"
server_key_iv: "909d32837c085bb083952c019aba5c50"
server_key_version: "encryptionkey@0"
I20260809 18:51:42.793135 25289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "20a5ae7efb1f42c7a9321043df56ea37"
format_stamp: "Formatted at 2026-08-09 18:51:42 on dist-test-slave-6zbq"
server_key: "828285449a1fa2f24ef8d7f6b3a65b4c"
server_key_iv: "909d32837c085bb083952c019aba5c50"
server_key_version: "encryptionkey@0"
I20260809 18:51:42.794390 25289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 18:51:42.795054 25311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:42.795199 25289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:42.795282 25289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "20a5ae7efb1f42c7a9321043df56ea37"
format_stamp: "Formatted at 2026-08-09 18:51:42 on dist-test-slave-6zbq"
server_key: "828285449a1fa2f24ef8d7f6b3a65b4c"
server_key_iv: "909d32837c085bb083952c019aba5c50"
server_key_version: "encryptionkey@0"
I20260809 18:51:42.795347 25289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:42.815667 25289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:42.817240 25289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:42.817360 25289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:42.820950 25289 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:34861
I20260809 18:51:42.820981 25397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:34861 every 8 connection(s)
I20260809 18:51:42.821274 25289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:51:42.822026 25398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:42.822610 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25289
I20260809 18:51:42.822739 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:51:42.822873 24427 external_mini_cluster.cc:1428] Setting key a8a8af6eb03588d864d2fddc998c7166
I20260809 18:51:42.824569 25398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37: Bootstrap starting.
I20260809 18:51:42.825110 25398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:42.825346 25398 log.cc:824] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:42.825955 25398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37: No bootstrap required, opened a new log
I20260809 18:51:42.827773 25398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a5ae7efb1f42c7a9321043df56ea37" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34861 } }
I20260809 18:51:42.827895 25398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:42.827929 25398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20a5ae7efb1f42c7a9321043df56ea37, State: Initialized, Role: FOLLOWER
I20260809 18:51:42.828048 25398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a5ae7efb1f42c7a9321043df56ea37" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34861 } }
I20260809 18:51:42.828135 25398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:51:42.828171 25398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:51:42.828217 25398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 18:51:42 dist-test-slave-6zbq krb5kdc[25271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:42.828830 25398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a5ae7efb1f42c7a9321043df56ea37" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34861 } }
I20260809 18:51:42.828931 25398 leader_election.cc:302] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20a5ae7efb1f42c7a9321043df56ea37; no voters:
I20260809 18:51:42.829104 25398 leader_election.cc:288] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:51:42.829130 25404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:42.829313 25404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [term 1 LEADER]: Becoming Leader. State: Replica: 20a5ae7efb1f42c7a9321043df56ea37, State: Running, Role: LEADER
I20260809 18:51:42.829432 25398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:51:42.829417 25404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a5ae7efb1f42c7a9321043df56ea37" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34861 } }
I20260809 18:51:42.829828 25404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20a5ae7efb1f42c7a9321043df56ea37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a5ae7efb1f42c7a9321043df56ea37" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34861 } } }
I20260809 18:51:42.829991 25404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:42.830161 25404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20a5ae7efb1f42c7a9321043df56ea37. Latest consensus state: current_term: 1 leader_uuid: "20a5ae7efb1f42c7a9321043df56ea37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a5ae7efb1f42c7a9321043df56ea37" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34861 } } }
I20260809 18:51:42.830277 25404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37 [sys.catalog]: This master's current role is: LEADER
W20260809 18:51:42.831198 25432 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:51:42.831256 25432 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:51:42.831303 25420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:51:42.831440 25420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:51:42.832433 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:42.823328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43664 (local address 127.23.218.254:34861)
0809 18:51:42.823503 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:42.823511 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:42.823534 (+ 23us) server_negotiation.cc:407] Connection header received
0809 18:51:42.823939 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:42.823946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:42.823992 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:42.824112 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:42.824715 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:42.825643 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:42.826433 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:42.826831 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:42.826924 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:42.829196 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:42.829200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:42.829200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:42.829210 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:42.831020 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:42.831369 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:42.831373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:42.831373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:42.831463 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:42.831650 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:42.831652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:42.831652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:42.831868 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:42.832011 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:42.832106 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 18:51:42.832245 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260809 18:51:42.832871 25420 catalog_manager.cc:1347] Generated new cluster ID: bf423fd2c228450dbb7f7ed3933f940a
I20260809 18:51:42.832922 25420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:51:42.858898 25420 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:51:42.859349 25420 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:51:42.868280 25420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20a5ae7efb1f42c7a9321043df56ea37: Generated new TSK 0
I20260809 18:51:42.868420 25420 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:51:42.895071 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34861
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:42.972831 25439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:42.973016 25439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:42.973049 25439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:42.974169 25439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:42.974206 25439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:42.974241 25439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:51:42.975749 25439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34861
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:42.975972 25439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:42.976163 25439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:42.980777 25447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:42.980775 25449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:42.980930 25445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:42.981021 25439 server_base.cc:1034] running on GCE node
I20260809 18:51:42.981220 25439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:42.981443 25439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:42.982585 25439 hybrid_clock.cc:648] HybridClock initialized: now 1786301502982571 us; error 32 us; skew 500 ppm
Aug 09 18:51:42 dist-test-slave-6zbq krb5kdc[25271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301502, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:42.984844 25439 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:51:42.985267 25439 webserver.cc:460] Webserver started at http://127.23.218.193:39309/ using document root <none> and password file <none>
I20260809 18:51:42.985463 25439 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:42.985510 25439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:42.985630 25439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:42.986716 25439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "3b501964907a417dad3754c9a2989f24"
format_stamp: "Formatted at 2026-08-09 18:51:42 on dist-test-slave-6zbq"
server_key: "c2042e031cdf962f3a38348e8f3a5bd3"
server_key_iv: "b19af13a4df25063acf41b87ad809219"
server_key_version: "encryptionkey@0"
I20260809 18:51:42.987027 25439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "3b501964907a417dad3754c9a2989f24"
format_stamp: "Formatted at 2026-08-09 18:51:42 on dist-test-slave-6zbq"
server_key: "c2042e031cdf962f3a38348e8f3a5bd3"
server_key_iv: "b19af13a4df25063acf41b87ad809219"
server_key_version: "encryptionkey@0"
I20260809 18:51:42.988229 25439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:51:42.988870 25461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:42.989010 25439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:42.989091 25439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "3b501964907a417dad3754c9a2989f24"
format_stamp: "Formatted at 2026-08-09 18:51:42 on dist-test-slave-6zbq"
server_key: "c2042e031cdf962f3a38348e8f3a5bd3"
server_key_iv: "b19af13a4df25063acf41b87ad809219"
server_key_version: "encryptionkey@0"
I20260809 18:51:42.989157 25439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:42.995924 25439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:42.997597 25439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:42.997725 25439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:42.997912 25439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:42.998180 25439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:42.998212 25439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:42.998237 25439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:42.998265 25439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:43.003659 25439 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:40597
I20260809 18:51:43.003700 25643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:40597 every 8 connection(s)
I20260809 18:51:43.003952 25439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:51:43.008970 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25439
I20260809 18:51:43.009052 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301502, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:43.009193 24427 external_mini_cluster.cc:1428] Setting key e82e042936f5bc0510121ea4a51071f9
I20260809 18:51:43.012277 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.004550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:39129 (local address 127.23.218.254:34861)
0809 18:51:43.004621 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.004623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.004769 (+ 146us) server_negotiation.cc:407] Connection header received
0809 18:51:43.005565 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.005566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.005576 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.005620 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:43.006129 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.006649 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.007545 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.007738 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.007778 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.009686 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:43.009688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:43.009688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:43.009695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:43.011026 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.011389 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.011391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.011391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.011448 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.011671 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.011672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.011672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.011773 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:43.011829 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.012074 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.012182 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 18:51:43.012377 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.004524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34861 (local address 127.23.218.193:39129)
0809 18:51:43.004667 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:43.004700 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:43.005489 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:43.005677 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:43.005681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:43.005899 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:43.006065 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.006068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.006711 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.006712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.007485 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.007488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.007787 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.007788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.007874 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.008230 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:43.008237 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:43.009597 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:43.011099 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:43.011105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:43.011107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:43.011313 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:43.011504 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:43.011505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:43.011505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:43.011605 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:43.011827 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:43.011834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:43.011959 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 18:51:43.012043 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 18:51:43.012184 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260809 18:51:43.012797 25645 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34861
I20260809 18:51:43.012881 25645 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:43.013034 25645 heartbeater.cc:511] Master 127.23.218.254:34861 requested a full tablet report, sending...
I20260809 18:51:43.013528 25341 ts_manager.cc:194] Registered new tserver with Master: 3b501964907a417dad3754c9a2989f24 (127.23.218.193:40597)
I20260809 18:51:43.014154 25341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:39129
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:51:43.030579 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34861
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:43.107228 25655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:43.107396 25655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:43.107427 25655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:43.108510 25655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:43.108546 25655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:43.108580 25655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:51:43.109983 25655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34861
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:43.110193 25655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:43.110415 25655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:43.114871 25663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:43.114952 25662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:43.114863 25665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:43.115025 25655 server_base.cc:1034] running on GCE node
I20260809 18:51:43.115224 25655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:43.115430 25655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:43.116551 25655 hybrid_clock.cc:648] HybridClock initialized: now 1786301503116539 us; error 26 us; skew 500 ppm
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301503, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:43.118808 25655 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:51:43.119195 25655 webserver.cc:460] Webserver started at http://127.23.218.194:46753/ using document root <none> and password file <none>
I20260809 18:51:43.119372 25655 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:43.119417 25655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:43.119531 25655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:43.120437 25655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "b3f8dba83a934c22bedbc26dd8065ab4"
format_stamp: "Formatted at 2026-08-09 18:51:43 on dist-test-slave-6zbq"
server_key: "d010c8b69b1e8f2e873037367a741afd"
server_key_iv: "29f6a44ce9b46f9684cdf15a77a87842"
server_key_version: "encryptionkey@0"
I20260809 18:51:43.120743 25655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "b3f8dba83a934c22bedbc26dd8065ab4"
format_stamp: "Formatted at 2026-08-09 18:51:43 on dist-test-slave-6zbq"
server_key: "d010c8b69b1e8f2e873037367a741afd"
server_key_iv: "29f6a44ce9b46f9684cdf15a77a87842"
server_key_version: "encryptionkey@0"
I20260809 18:51:43.121892 25655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:51:43.122515 25674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:43.122655 25655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:43.122732 25655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "b3f8dba83a934c22bedbc26dd8065ab4"
format_stamp: "Formatted at 2026-08-09 18:51:43 on dist-test-slave-6zbq"
server_key: "d010c8b69b1e8f2e873037367a741afd"
server_key_iv: "29f6a44ce9b46f9684cdf15a77a87842"
server_key_version: "encryptionkey@0"
I20260809 18:51:43.122795 25655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:43.134701 25655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:43.136530 25655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:43.136652 25655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:43.136829 25655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:43.137075 25655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:43.137116 25655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:43.137141 25655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:43.137166 25655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:43.142467 25655 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:37543
I20260809 18:51:43.142527 25862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:37543 every 8 connection(s)
I20260809 18:51:43.142762 25655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:51:43.144438 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25655
I20260809 18:51:43.144513 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:51:43.144668 24427 external_mini_cluster.cc:1428] Setting key fa3ae29cb134a504ad1a1d1c505e30d7
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301503, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:43.151439 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.143358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:45403 (local address 127.23.218.254:34861)
0809 18:51:43.143407 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.143410 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.143591 (+ 181us) server_negotiation.cc:407] Connection header received
0809 18:51:43.144429 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.144431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.144444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.144494 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:43.145130 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.145751 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.146755 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.146953 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.147009 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.148995 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:43.148996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:43.148997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:43.149004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:43.150203 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.150597 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.150598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.150599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.150654 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.150868 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.150869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.150869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.150974 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:43.151029 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.151271 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.151352 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:51:43.151563 25865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.143327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34861 (local address 127.23.218.194:45403)
0809 18:51:43.143487 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:43.143518 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:43.144333 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:43.144558 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:43.144562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:43.144839 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:43.145054 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.145060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.145838 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.145839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.146665 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.146668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.147021 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.147022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.147139 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.147541 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:43.147548 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:43.148908 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:43.150274 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:43.150281 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:43.150283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:43.150532 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:43.150713 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:43.150714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:43.150714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:43.150814 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:43.151027 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:43.151033 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:43.151156 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 18:51:43.151242 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 18:51:43.151378 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260809 18:51:43.151959 25863 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34861
I20260809 18:51:43.152036 25863 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:43.152194 25863 heartbeater.cc:511] Master 127.23.218.254:34861 requested a full tablet report, sending...
I20260809 18:51:43.152516 25341 ts_manager.cc:194] Registered new tserver with Master: b3f8dba83a934c22bedbc26dd8065ab4 (127.23.218.194:37543)
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
I20260809 18:51:43.152866 25341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:45403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:51:43.166227 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34861
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:43.243810 25873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:43.243988 25873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:43.244009 25873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:43.245126 25873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:43.245164 25873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:43.245189 25873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:51:43.246675 25873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34861
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:43.246881 25873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:43.247064 25873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:43.251658 25886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:43.251693 25883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:43.251660 25882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:43.251684 25873 server_base.cc:1034] running on GCE node
I20260809 18:51:43.252064 25873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:43.252291 25873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:43.253420 25873 hybrid_clock.cc:648] HybridClock initialized: now 1786301503253408 us; error 29 us; skew 500 ppm
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301503, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:43.255714 25873 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:51:43.256112 25873 webserver.cc:460] Webserver started at http://127.23.218.195:33651/ using document root <none> and password file <none>
I20260809 18:51:43.256295 25873 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:43.256340 25873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:43.256459 25873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:43.257413 25873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "e18285284dc04b60bb4f6e17c99f7687"
format_stamp: "Formatted at 2026-08-09 18:51:43 on dist-test-slave-6zbq"
server_key: "359321aaf51f8b29ecfcbf9c2a79a58f"
server_key_iv: "242bea4789444447398a340c59031d28"
server_key_version: "encryptionkey@0"
I20260809 18:51:43.257802 25873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "e18285284dc04b60bb4f6e17c99f7687"
format_stamp: "Formatted at 2026-08-09 18:51:43 on dist-test-slave-6zbq"
server_key: "359321aaf51f8b29ecfcbf9c2a79a58f"
server_key_iv: "242bea4789444447398a340c59031d28"
server_key_version: "encryptionkey@0"
I20260809 18:51:43.259018 25873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:51:43.259707 25898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:43.259845 25873 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:43.259927 25873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "e18285284dc04b60bb4f6e17c99f7687"
format_stamp: "Formatted at 2026-08-09 18:51:43 on dist-test-slave-6zbq"
server_key: "359321aaf51f8b29ecfcbf9c2a79a58f"
server_key_iv: "242bea4789444447398a340c59031d28"
server_key_version: "encryptionkey@0"
I20260809 18:51:43.259982 25873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:43.267845 25873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:43.269331 25873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:43.269450 25873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:43.269640 25873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:43.269917 25873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:43.269950 25873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:43.269975 25873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:43.269992 25873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:43.275748 25873 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:43773
I20260809 18:51:43.275802 26070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:43773 every 8 connection(s)
I20260809 18:51:43.276052 25873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:51:43.280097 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25873
I20260809 18:51:43.280203 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:51:43.280352 24427 external_mini_cluster.cc:1428] Setting key 1fb90b80df35a103c6d695b600538fa5
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301503, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:43.283946 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.276675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:50645 (local address 127.23.218.254:34861)
0809 18:51:43.276724 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.276726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.276888 (+ 162us) server_negotiation.cc:407] Connection header received
0809 18:51:43.277707 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.277708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.277717 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.277760 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:43.278271 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.278804 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.279711 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.279882 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.279937 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.281938 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:43.281939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:43.281939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:43.281945 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:43.283007 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.283325 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.283326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.283326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.283370 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.283522 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.283522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.283522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.283608 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:43.283651 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.283837 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.283887 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:51:43.284093 26076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.276639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34861 (local address 127.23.218.195:50645)
0809 18:51:43.276783 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:43.276816 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:43.277627 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:43.277814 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:43.277817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:43.278041 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:43.278213 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.278218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.278868 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.278869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.279652 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.279654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.279945 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.279946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.280080 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.280525 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:43.280533 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:43.281863 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:43.283068 (+ 1205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:43.283075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:43.283076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:43.283270 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:43.283405 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:43.283405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:43.283406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:43.283483 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:43.283654 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:43.283657 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:43.283752 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 18:51:43.283823 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 18:51:43.283927 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260809 18:51:43.284512 26071 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34861
I20260809 18:51:43.284592 26071 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:43.284746 26071 heartbeater.cc:511] Master 127.23.218.254:34861 requested a full tablet report, sending...
I20260809 18:51:43.285077 25341 ts_manager.cc:194] Registered new tserver with Master: e18285284dc04b60bb4f6e17c99f7687 (127.23.218.195:43773)
I20260809 18:51:43.285424 25341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:50645
I20260809 18:51:43.291838 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:51:43.298105 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.293284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.23.218.254:34861)
0809 18:51:43.293327 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.293329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.293428 (+ 99us) server_negotiation.cc:407] Connection header received
0809 18:51:43.293468 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.293468 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.293476 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.293516 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:43.293962 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.294411 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.295080 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.295254 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.295303 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.296277 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:43.296277 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:43.296278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:43.296282 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:43.297293 (+ 1011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.297580 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.297580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.297581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.297624 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.297752 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.297752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.297752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.297859 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:43.297925 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.298003 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.298054 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:51:43.300051 25341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:51:43.301515 25341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:51:43.307322 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.303726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:37543 (local address 127.0.0.1:37000)
0809 18:51:43.304059 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:43.304067 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:43.304122 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:43.304394 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:43.304395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:43.304397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:43.304439 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.304440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.305257 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.305258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.306091 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.306094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.307017 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.307018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.307111 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.307126 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 18:51:43.307201 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 18:51:43.307253 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":29,"thread_start_us":59,"threads_started":1}
I20260809 18:51:43.307418 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.303422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40597 (local address 127.0.0.1:50880)
0809 18:51:43.303523 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:43.303558 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:43.303621 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:43.303881 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:43.303883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:43.303886 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:43.303943 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.303944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.305217 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.305219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.306508 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.306511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.307235 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.307236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.307295 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.307298 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:51:43.307325 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 18:51:43.307355 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":54,"threads_started":1}
I20260809 18:51:43.307343 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.303850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37000 (local address 127.23.218.194:37543)
0809 18:51:43.304014 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.304019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.304170 (+ 151us) server_negotiation.cc:407] Connection header received
0809 18:51:43.304213 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.304216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.304232 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.304298 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:43.304513 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.305173 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.306186 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.306932 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.307007 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.307056 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.307221 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.307266 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260809 18:51:43.307619 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.303599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:43773 (local address 127.0.0.1:51148)
0809 18:51:43.304007 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:43.304017 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:43.304069 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:43.304308 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:43.304309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:43.304311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:43.304355 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.304356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.305471 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.305472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.306549 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.306551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.307396 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.307397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.307467 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.307471 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:51:43.307514 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 18:51:43.307553 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":47,"threads_started":1}
I20260809 18:51:43.307744 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.303537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50880 (local address 127.23.218.193:40597)
0809 18:51:43.303741 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.303745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.303754 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:51:43.303791 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.303793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.303807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.303876 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:43.304008 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.305111 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.306617 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.307149 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.307517 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.307597 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.307625 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.307669 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":79,"threads_started":1}
I20260809 18:51:43.307870 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.303840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.23.218.195:43773)
0809 18:51:43.304069 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.304074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.304083 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:51:43.304118 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.304120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.304133 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.304239 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:43.304422 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.305380 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.306620 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.307248 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.307312 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.307359 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.307767 (+ 408us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.307807 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":79,"threads_started":1}
I20260809 18:51:43.308308 25752 tablet_service.cc:1467] Processing CreateTablet for tablet 143eeb611a3e47cca33c5ecaf160a5dc (DEFAULT_TABLE table=test-table [id=f3bdd420f3674e1e8ef6fa5c6a926167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:43.308499 25540 tablet_service.cc:1467] Processing CreateTablet for tablet 143eeb611a3e47cca33c5ecaf160a5dc (DEFAULT_TABLE table=test-table [id=f3bdd420f3674e1e8ef6fa5c6a926167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:43.308483 25981 tablet_service.cc:1467] Processing CreateTablet for tablet 143eeb611a3e47cca33c5ecaf160a5dc (DEFAULT_TABLE table=test-table [id=f3bdd420f3674e1e8ef6fa5c6a926167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:43.308607 25752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 143eeb611a3e47cca33c5ecaf160a5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:43.308780 25981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 143eeb611a3e47cca33c5ecaf160a5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:43.308780 25540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 143eeb611a3e47cca33c5ecaf160a5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:43.310961 26099 tablet_bootstrap.cc:492] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4: Bootstrap starting.
I20260809 18:51:43.311518 26099 tablet_bootstrap.cc:654] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:43.311846 26103 tablet_bootstrap.cc:492] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687: Bootstrap starting.
I20260809 18:51:43.312077 26101 tablet_bootstrap.cc:492] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24: Bootstrap starting.
I20260809 18:51:43.312237 26099 log.cc:824] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:43.312695 26101 tablet_bootstrap.cc:654] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:43.312944 26099 tablet_bootstrap.cc:492] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4: No bootstrap required, opened a new log
I20260809 18:51:43.313017 26099 ts_tablet_manager.cc:1397] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:51:43.313237 26101 log.cc:824] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:43.313864 26101 tablet_bootstrap.cc:492] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24: No bootstrap required, opened a new log
I20260809 18:51:43.313925 26101 ts_tablet_manager.cc:1397] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:51:43.314026 26103 tablet_bootstrap.cc:654] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:43.314606 26103 log.cc:824] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:43.314991 26099 raft_consensus.cc:348] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } }
I20260809 18:51:43.315156 26099 raft_consensus.cc:374] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:43.315191 26099 raft_consensus.cc:729] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3f8dba83a934c22bedbc26dd8065ab4, State: Initialized, Role: FOLLOWER
I20260809 18:51:43.315331 26103 tablet_bootstrap.cc:492] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687: No bootstrap required, opened a new log
I20260809 18:51:43.315321 26099 consensus_queue.cc:260] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } }
I20260809 18:51:43.315404 26103 ts_tablet_manager.cc:1397] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 18:51:43.315572 26099 ts_tablet_manager.cc:1428] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:51:43.315586 25863 heartbeater.cc:503] Master 127.23.218.254:34861 was elected leader, sending a full tablet report...
I20260809 18:51:43.315766 26101 raft_consensus.cc:348] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } }
I20260809 18:51:43.315929 26101 raft_consensus.cc:374] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:43.315968 26101 raft_consensus.cc:729] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b501964907a417dad3754c9a2989f24, State: Initialized, Role: FOLLOWER
I20260809 18:51:43.316071 26101 consensus_queue.cc:260] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } }
I20260809 18:51:43.316321 26101 ts_tablet_manager.cc:1428] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 18:51:43.316462 25645 heartbeater.cc:503] Master 127.23.218.254:34861 was elected leader, sending a full tablet report...
I20260809 18:51:43.317157 26103 raft_consensus.cc:348] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } }
I20260809 18:51:43.317289 26103 raft_consensus.cc:374] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:43.317309 26103 raft_consensus.cc:729] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18285284dc04b60bb4f6e17c99f7687, State: Initialized, Role: FOLLOWER
I20260809 18:51:43.317410 26103 consensus_queue.cc:260] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } }
I20260809 18:51:43.317646 26103 ts_tablet_manager.cc:1428] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 18:51:43.317684 26071 heartbeater.cc:503] Master 127.23.218.254:34861 was elected leader, sending a full tablet report...
W20260809 18:51:43.393222 25864 tablet.cc:2369] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:51:43.483955 26109 raft_consensus.cc:482] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:43.484079 26109 raft_consensus.cc:504] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } }
I20260809 18:51:43.484419 26109 leader_election.cc:288] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b501964907a417dad3754c9a2989f24 (127.23.218.193:40597), e18285284dc04b60bb4f6e17c99f7687 (127.23.218.195:43773)
I20260809 18:51:43.487098 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.484685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:59111 (local address 127.23.218.193:40597)
0809 18:51:43.484730 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.484732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.484833 (+ 101us) server_negotiation.cc:407] Connection header received
0809 18:51:43.484862 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.484862 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.484872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.484918 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:43.485054 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.485515 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.486444 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.486803 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.486864 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.486873 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.487000 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.487035 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:51:43.487097 25865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.484635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40597 (local address 127.23.218.194:59111)
0809 18:51:43.484709 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:43.484721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:43.484782 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:43.484925 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:43.484926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:43.484928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:43.484995 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.484997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.485588 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.485588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.486366 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.486369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.486874 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.486875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.486940 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.486944 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:51:43.486989 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 18:51:43.487027 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 18:51:43.487330 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.484738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:43773 (local address 127.23.218.194:37417)
0809 18:51:43.484901 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:43.484910 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:43.484967 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:43.485101 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:43.485102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:43.485103 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:43.485175 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.485177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.485775 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.485775 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.486562 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:43.486564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.487094 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.487095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:43.487161 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.487165 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:51:43.487216 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 18:51:43.487257 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260809 18:51:43.487457 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.484821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:37417 (local address 127.23.218.195:43773)
0809 18:51:43.484862 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.484864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.484964 (+ 100us) server_negotiation.cc:407] Connection header received
0809 18:51:43.485021 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.485022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.485032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.485094 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:43.485235 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.485701 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.486647 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.487020 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.487088 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.487096 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.487286 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.487327 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:51:43.487413 25576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "143eeb611a3e47cca33c5ecaf160a5dc" candidate_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b501964907a417dad3754c9a2989f24" is_pre_election: true
I20260809 18:51:43.487568 25576 raft_consensus.cc:2454] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3f8dba83a934c22bedbc26dd8065ab4 in term 0.
I20260809 18:51:43.487699 26004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "143eeb611a3e47cca33c5ecaf160a5dc" candidate_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18285284dc04b60bb4f6e17c99f7687" is_pre_election: true
I20260809 18:51:43.487797 25678 leader_election.cc:302] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b501964907a417dad3754c9a2989f24, b3f8dba83a934c22bedbc26dd8065ab4; no voters:
I20260809 18:51:43.487864 26004 raft_consensus.cc:2454] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3f8dba83a934c22bedbc26dd8065ab4 in term 0.
I20260809 18:51:43.487914 26109 raft_consensus.cc:2781] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:51:43.487989 26109 raft_consensus.cc:482] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:51:43.488015 26109 raft_consensus.cc:3037] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:43.488698 26109 raft_consensus.cc:504] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } }
I20260809 18:51:43.488809 26109 leader_election.cc:288] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 3b501964907a417dad3754c9a2989f24 (127.23.218.193:40597), e18285284dc04b60bb4f6e17c99f7687 (127.23.218.195:43773)
I20260809 18:51:43.488971 25576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "143eeb611a3e47cca33c5ecaf160a5dc" candidate_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b501964907a417dad3754c9a2989f24"
I20260809 18:51:43.489007 26004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "143eeb611a3e47cca33c5ecaf160a5dc" candidate_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e18285284dc04b60bb4f6e17c99f7687"
I20260809 18:51:43.489061 25576 raft_consensus.cc:3037] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:43.489068 26004 raft_consensus.cc:3037] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:43.489928 25576 raft_consensus.cc:2454] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3f8dba83a934c22bedbc26dd8065ab4 in term 1.
I20260809 18:51:43.489928 26004 raft_consensus.cc:2454] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3f8dba83a934c22bedbc26dd8065ab4 in term 1.
I20260809 18:51:43.490093 25678 leader_election.cc:302] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b501964907a417dad3754c9a2989f24, b3f8dba83a934c22bedbc26dd8065ab4; no voters:
I20260809 18:51:43.490175 26109 raft_consensus.cc:2781] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:43.490307 26109 raft_consensus.cc:686] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [term 1 LEADER]: Becoming Leader. State: Replica: b3f8dba83a934c22bedbc26dd8065ab4, State: Running, Role: LEADER
I20260809 18:51:43.490438 26109 consensus_queue.cc:237] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } }
I20260809 18:51:43.491215 25341 catalog_manager.cc:5495] T 143eeb611a3e47cca33c5ecaf160a5dc P b3f8dba83a934c22bedbc26dd8065ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3f8dba83a934c22bedbc26dd8065ab4 (127.23.218.194). New cstate: current_term: 1 leader_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b501964907a417dad3754c9a2989f24" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e18285284dc04b60bb4f6e17c99f7687" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3f8dba83a934c22bedbc26dd8065ab4" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 37543 } health_report { overall_health: HEALTHY } } }
W20260809 18:51:43.504580 25646 tablet.cc:2369] T 143eeb611a3e47cca33c5ecaf160a5dc P 3b501964907a417dad3754c9a2989f24: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:51:43.526520 26074 tablet.cc:2369] T 143eeb611a3e47cca33c5ecaf160a5dc P e18285284dc04b60bb4f6e17c99f7687: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:51:43.536871 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.530318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.23.218.193:40597)
0809 18:51:43.530393 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.530396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.530493 (+ 97us) server_negotiation.cc:407] Connection header received
0809 18:51:43.530556 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.530557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.530567 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.530617 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:43.531163 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.531596 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.532255 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.532431 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.532484 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.534612 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:43.534621 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:43.534621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:43.534628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:43.535845 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.536160 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.536163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.536164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.536224 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.536387 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.536388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.536388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.536513 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:43.536639 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.536710 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.536789 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301503, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301503, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:51:43.558063 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.552280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50906 (local address 127.23.218.193:40597)
0809 18:51:43.552329 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.552331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.552393 (+ 62us) server_negotiation.cc:407] Connection header received
0809 18:51:43.552457 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.552458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.552467 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.552510 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:43.552962 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.553390 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.554038 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.554219 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.554270 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.556127 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:43.556129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:43.556129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:43.556136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:43.557119 (+ 983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.557396 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.557397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.557397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.557442 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.557596 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.557597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.557597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.557691 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:43.557771 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.557915 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.557985 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[25271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:51:43.577625 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:43.572105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50920 (local address 127.23.218.193:40597)
0809 18:51:43.572148 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:43.572150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:43.572210 (+ 60us) server_negotiation.cc:407] Connection header received
0809 18:51:43.572260 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:43.572260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:43.572269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:43.572313 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:43.572752 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.573168 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.573798 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:43.573968 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:43.574018 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:43.575782 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:43.575783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:43.575783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:43.575790 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:43.576740 (+ 950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.577008 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.577009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.577009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.577053 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:43.577218 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:43.577219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:43.577219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:43.577300 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:43.577378 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:43.577483 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 18:51:43.577549 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 18:51:43.578027 25506 tablet_service.cc:2908] Rejecting scan request for tablet 143eeb611a3e47cca33c5ecaf160a5dc: Uninitialized: clean time has not yet been initialized
I20260809 18:51:43.578691 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25439
I20260809 18:51:43.590305 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25655
I20260809 18:51:43.599524 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25873
I20260809 18:51:43.606087 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25289
2026-08-09T18:51:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2142 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[26138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[26138](info): set up 2 sockets
Aug 09 18:51:43 dist-test-slave-6zbq krb5kdc[26138](info): commencing operation
krb5kdc: starting...
W20260809 18:51:44.741546 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:51:44 dist-test-slave-6zbq krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:51:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:51:44.844421 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34539
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:34539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:44.921811 26164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:44.921985 26164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:44.922005 26164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:44.923113 26164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:51:44.923151 26164 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 18:51:44.923162 26164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:44.923174 26164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:44.923189 26164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:51:44.923199 26164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:51:44.924576 26164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:34539
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:44.924757 26164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:44.924913 26164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:44.929226 26178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:44.929204 26180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:44.929333 26164 server_base.cc:1034] running on GCE node
W20260809 18:51:44.929209 26177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:44.929569 26164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:44.929814 26164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:44.930943 26164 hybrid_clock.cc:648] HybridClock initialized: now 1786301504930930 us; error 29 us; skew 500 ppm
Aug 09 18:51:44 dist-test-slave-6zbq krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:44.933118 26164 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:51:44.933498 26164 webserver.cc:460] Webserver started at http://127.23.218.254:34253/ using document root <none> and password file <none>
I20260809 18:51:44.933678 26164 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:44.933722 26164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:44.933836 26164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:44.934816 26164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "0d0565e5282e4022aa383150d02d277a"
format_stamp: "Formatted at 2026-08-09 18:51:44 on dist-test-slave-6zbq"
server_key: "d30d4ae18c46b5fab15d14cf6e60f852"
server_key_iv: "577622748b0b8d8b2545f178bcba9e45"
server_key_version: "encryptionkey@0"
I20260809 18:51:44.935132 26164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "0d0565e5282e4022aa383150d02d277a"
format_stamp: "Formatted at 2026-08-09 18:51:44 on dist-test-slave-6zbq"
server_key: "d30d4ae18c46b5fab15d14cf6e60f852"
server_key_iv: "577622748b0b8d8b2545f178bcba9e45"
server_key_version: "encryptionkey@0"
I20260809 18:51:44.936277 26164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:51:44.936869 26193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:44.937013 26164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:44.937093 26164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "0d0565e5282e4022aa383150d02d277a"
format_stamp: "Formatted at 2026-08-09 18:51:44 on dist-test-slave-6zbq"
server_key: "d30d4ae18c46b5fab15d14cf6e60f852"
server_key_iv: "577622748b0b8d8b2545f178bcba9e45"
server_key_version: "encryptionkey@0"
I20260809 18:51:44.937156 26164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:44.976859 26164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:44.978708 26164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:44.978817 26164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:44.983217 26164 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:34539
I20260809 18:51:44.983309 26303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:34539 every 8 connection(s)
I20260809 18:51:44.983572 26164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:51:44.984210 26304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:44.986465 26304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a: Bootstrap starting.
I20260809 18:51:44.987022 26304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:44.987272 26304 log.cc:824] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:44.987890 26304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a: No bootstrap required, opened a new log
I20260809 18:51:44.988404 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 26164
I20260809 18:51:44.988483 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:51:44.988624 24427 external_mini_cluster.cc:1428] Setting key f92760cba66c9fd09b773ee5444ad278
I20260809 18:51:44.989871 26304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d0565e5282e4022aa383150d02d277a" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34539 } }
I20260809 18:51:44.990044 26304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:44.990145 26304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d0565e5282e4022aa383150d02d277a, State: Initialized, Role: FOLLOWER
I20260809 18:51:44.990283 26304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d0565e5282e4022aa383150d02d277a" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34539 } }
I20260809 18:51:44.990387 26304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:51:44.990423 26304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:51:44.990466 26304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:44.991086 26304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d0565e5282e4022aa383150d02d277a" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34539 } }
I20260809 18:51:44.991192 26304 leader_election.cc:302] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d0565e5282e4022aa383150d02d277a; no voters:
I20260809 18:51:44.991345 26304 leader_election.cc:288] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:51:44.991384 26310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:44.991523 26310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [term 1 LEADER]: Becoming Leader. State: Replica: 0d0565e5282e4022aa383150d02d277a, State: Running, Role: LEADER
I20260809 18:51:44.991564 26304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:51:44.991675 26310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d0565e5282e4022aa383150d02d277a" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34539 } }
I20260809 18:51:44.992224 26314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d0565e5282e4022aa383150d02d277a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d0565e5282e4022aa383150d02d277a" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34539 } } }
I20260809 18:51:44.992314 26314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:44.992453 26313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d0565e5282e4022aa383150d02d277a. Latest consensus state: current_term: 1 leader_uuid: "0d0565e5282e4022aa383150d02d277a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d0565e5282e4022aa383150d02d277a" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34539 } } }
I20260809 18:51:44.992498 26313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:44.992792 26331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:51:44.993276 26331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:51:44.994897 26331 catalog_manager.cc:1347] Generated new cluster ID: 7eaf243fc4d94fd9a03355b63041f35f
I20260809 18:51:44.994951 26331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:51:44 dist-test-slave-6zbq krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:44.999910 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:44.989071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46256 (local address 127.23.218.254:34539)
0809 18:51:44.989258 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:44.989269 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:44.989304 (+ 35us) server_negotiation.cc:407] Connection header received
0809 18:51:44.989827 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:44.989835 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:44.989898 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:44.990043 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:44.990741 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:44.992022 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:44.993028 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:44.993593 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:44.993712 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:44.996290 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:44.996294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:44.996295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:44.996306 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:44.998251 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:44.998677 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:44.998680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:44.998680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:44.998796 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:44.998989 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:44.998990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:44.998991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:44.999251 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:44.999384 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:44.999487 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 18:51:44.999713 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260809 18:51:45.009655 26331 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:51:45.010095 26331 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:51:45.012940 26331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a: Generated new TSK 0
I20260809 18:51:45.013085 26331 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:51:45.043816 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34539
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:45.122511 26352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:45.122687 26352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:45.122717 26352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:45.123807 26352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:45.123845 26352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:45.123871 26352 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 18:51:45.123894 26352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:51:45.125275 26352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.218.254:34539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:45.125483 26352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:45.125679 26352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:45.130254 26360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:45.130263 26359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:45.130276 26362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:45.130461 26352 server_base.cc:1034] running on GCE node
I20260809 18:51:45.130683 26352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:45.130900 26352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:45.132040 26352 hybrid_clock.cc:648] HybridClock initialized: now 1786301505132026 us; error 32 us; skew 500 ppm
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:45.134286 26352 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:51:45.134703 26352 webserver.cc:460] Webserver started at http://127.23.218.193:42171/ using document root <none> and password file <none>
I20260809 18:51:45.134882 26352 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:45.134923 26352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:45.135042 26352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:45.135970 26352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "5983bbff72c7496e8a3c85dd2758eb35"
format_stamp: "Formatted at 2026-08-09 18:51:45 on dist-test-slave-6zbq"
server_key: "ffb7e8143f7aa09e55aa5f5d00c4303f"
server_key_iv: "0b68e770805eb67b96da70444476ff0c"
server_key_version: "encryptionkey@0"
I20260809 18:51:45.136288 26352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "5983bbff72c7496e8a3c85dd2758eb35"
format_stamp: "Formatted at 2026-08-09 18:51:45 on dist-test-slave-6zbq"
server_key: "ffb7e8143f7aa09e55aa5f5d00c4303f"
server_key_iv: "0b68e770805eb67b96da70444476ff0c"
server_key_version: "encryptionkey@0"
I20260809 18:51:45.137401 26352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:51:45.137993 26376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.138116 26352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.138202 26352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "5983bbff72c7496e8a3c85dd2758eb35"
format_stamp: "Formatted at 2026-08-09 18:51:45 on dist-test-slave-6zbq"
server_key: "ffb7e8143f7aa09e55aa5f5d00c4303f"
server_key_iv: "0b68e770805eb67b96da70444476ff0c"
server_key_version: "encryptionkey@0"
I20260809 18:51:45.138264 26352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:45.150458 26352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:45.151911 26352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:45.152024 26352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:45.152464 26352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:45.152503 26352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.152520 26352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:45.152531 26352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:45.159078 26352 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:40567
I20260809 18:51:45.159150 26559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:40567 every 8 connection(s)
I20260809 18:51:45.159382 26352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:51:45.161586 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.152772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:46251 (local address 127.23.218.254:34539)
0809 18:51:45.152828 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.152831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.153006 (+ 175us) server_negotiation.cc:407] Connection header received
0809 18:51:45.153406 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.153408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.153421 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.153498 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.154055 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.154809 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.155769 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.156063 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.156144 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.159044 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.159046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.159046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.159054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.160375 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.160727 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.160729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.160729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.160782 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.160992 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.160993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.160993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.161111 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.161164 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.161409 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.161496 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:51:45.161705 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.152764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34539 (local address 127.23.218.193:46251)
0809 18:51:45.152897 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.152932 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.153325 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.153548 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.153551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.153790 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:45.153983 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.153988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.154900 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.154901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.155699 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.155701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.156168 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.156169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.156288 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.156730 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:45.156740 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:45.158960 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:45.160449 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.160455 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.160457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.160655 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.160836 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.160837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.160837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.160938 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.161168 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:45.161174 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:45.161296 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.161378 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.161514 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":72,"threads_started":1}
I20260809 18:51:45.162103 26561 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34539
I20260809 18:51:45.162180 26561 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:45.162324 26561 heartbeater.cc:511] Master 127.23.218.254:34539 requested a full tablet report, sending...
I20260809 18:51:45.162818 26224 ts_manager.cc:194] Registered new tserver with Master: 5983bbff72c7496e8a3c85dd2758eb35 (127.23.218.193:40567)
I20260809 18:51:45.163456 26224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:46251
I20260809 18:51:45.167425 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.163015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34539 (local address 127.23.218.193:58701)
0809 18:51:45.163137 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.163146 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.163199 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.163315 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.163316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.163472 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:45.163527 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.163529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.164117 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.164118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.164626 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.164628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.164939 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.164940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.165008 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.165317 (+ 309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:45.165320 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:45.165570 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:45.166643 (+ 1073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.166644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.166645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.166822 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.166993 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.166994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.166994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.167041 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.167214 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:45.167215 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:45.167258 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.167302 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.167365 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
I20260809 18:51:45.167436 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.163063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:58701 (local address 127.23.218.254:34539)
0809 18:51:45.163094 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.163095 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.163201 (+ 106us) server_negotiation.cc:407] Connection header received
0809 18:51:45.163251 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.163252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.163260 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.163311 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.163594 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.164035 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.164694 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.164884 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.164935 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.165644 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.165645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.165646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.165652 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.166578 (+ 926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.166887 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.166888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.166888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.166939 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.167082 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.167082 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.167082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.167172 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.167216 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.167323 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.167381 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 18:51:45.168787 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 26352
I20260809 18:51:45.168867 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:51:45.169014 24427 external_mini_cluster.cc:1428] Setting key d59dc23e15508ab47f8075772aee1a15
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:51:45.190099 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34539
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:45.266570 26573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:45.266731 26573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:45.266752 26573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:45.267838 26573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:45.267876 26573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:45.267895 26573 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 18:51:45.267913 26573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:51:45.269330 26573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.218.254:34539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:45.269526 26573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:45.269709 26573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:45.274214 26581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:45.274238 26585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:45.274232 26580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:45.274329 26573 server_base.cc:1034] running on GCE node
I20260809 18:51:45.274587 26573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:45.274808 26573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:45.275940 26573 hybrid_clock.cc:648] HybridClock initialized: now 1786301505275918 us; error 34 us; skew 500 ppm
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:45.278131 26573 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:51:45.278539 26573 webserver.cc:460] Webserver started at http://127.23.218.194:33531/ using document root <none> and password file <none>
I20260809 18:51:45.278708 26573 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:45.278745 26573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:45.278827 26573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:45.279678 26573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "213496751f864fceb4bc29e000c3fa22"
format_stamp: "Formatted at 2026-08-09 18:51:45 on dist-test-slave-6zbq"
server_key: "5763cd6029a67a593fa435c1a7a762b0"
server_key_iv: "70b640c71846704b6587ed126cf414a4"
server_key_version: "encryptionkey@0"
I20260809 18:51:45.279944 26573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "213496751f864fceb4bc29e000c3fa22"
format_stamp: "Formatted at 2026-08-09 18:51:45 on dist-test-slave-6zbq"
server_key: "5763cd6029a67a593fa435c1a7a762b0"
server_key_iv: "70b640c71846704b6587ed126cf414a4"
server_key_version: "encryptionkey@0"
I20260809 18:51:45.281023 26573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:51:45.281625 26598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.281782 26573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.281862 26573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "213496751f864fceb4bc29e000c3fa22"
format_stamp: "Formatted at 2026-08-09 18:51:45 on dist-test-slave-6zbq"
server_key: "5763cd6029a67a593fa435c1a7a762b0"
server_key_iv: "70b640c71846704b6587ed126cf414a4"
server_key_version: "encryptionkey@0"
I20260809 18:51:45.281912 26573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:45.294540 26573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:45.296023 26573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:45.296118 26573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:45.296504 26573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:45.296540 26573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.296556 26573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:45.296567 26573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:45.302670 26573 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:46747
I20260809 18:51:45.302732 26782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:46747 every 8 connection(s)
I20260809 18:51:45.303016 26573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:51:45.303839 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 26573
I20260809 18:51:45.303908 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:51:45.304064 24427 external_mini_cluster.cc:1428] Setting key 7d49e74a038c5073158e1feb8d8d489a
I20260809 18:51:45.305399 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.296887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:37129 (local address 127.23.218.254:34539)
0809 18:51:45.296947 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.296949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.297199 (+ 250us) server_negotiation.cc:407] Connection header received
0809 18:51:45.297603 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.297605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.297617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.297690 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.298430 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.298923 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.300307 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.300599 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.300684 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.302820 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.302822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.302822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.302829 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.304212 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.304547 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.304547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.304548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.304592 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.304812 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.304812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.304813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.304920 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.304983 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.305235 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.305313 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 18:51:45.305531 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.296854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34539 (local address 127.23.218.194:37129)
0809 18:51:45.297007 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.297041 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.297502 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.297755 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.297759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.298066 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:45.298325 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.298332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.299004 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.299006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.300214 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.300217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.300726 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.300727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.300851 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.301300 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:45.301307 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:45.302736 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:45.304285 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.304291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.304293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.304491 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.304653 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.304654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.304654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.304757 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.304983 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:45.304990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:45.305115 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.305205 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.305344 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260809 18:51:45.306003 26785 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34539
I20260809 18:51:45.306092 26785 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:45.306254 26785 heartbeater.cc:511] Master 127.23.218.254:34539 requested a full tablet report, sending...
I20260809 18:51:45.306653 26224 ts_manager.cc:194] Registered new tserver with Master: 213496751f864fceb4bc29e000c3fa22 (127.23.218.194:46747)
I20260809 18:51:45.307015 26224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:37129
I20260809 18:51:45.312880 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.307805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:40887 (local address 127.23.218.254:34539)
0809 18:51:45.307837 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.307839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.307959 (+ 120us) server_negotiation.cc:407] Connection header received
0809 18:51:45.308004 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.308005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.308012 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.308055 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.308357 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.308789 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.309722 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.309897 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.309944 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.310817 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.310818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.310818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.310823 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.311922 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.312259 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.312260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.312260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.312300 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.312468 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.312468 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.312468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.312580 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.312624 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.312777 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.312827 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 18:51:45.312932 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.307748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34539 (local address 127.23.218.194:40887)
0809 18:51:45.307893 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.307902 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.307959 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.308060 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.308061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.308231 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:45.308293 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.308295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.308882 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.308883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.309634 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.309637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.309963 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.309964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.310035 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.310394 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:45.310399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:45.310743 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:45.311998 (+ 1255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.311999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.312000 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.312193 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.312354 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.312355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.312355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.312412 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.312635 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:45.312636 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:45.312690 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.312759 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.312848 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:51:45.326558 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34539
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:45.405956 26797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:45.406157 26797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:45.406189 26797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:45.407371 26797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:45.407411 26797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:45.407438 26797 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 18:51:45.407460 26797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:51:45.408870 26797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.218.254:34539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:45.409081 26797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:45.409282 26797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:45.413816 26812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:45.413826 26807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:45.413856 26806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:45.414041 26797 server_base.cc:1034] running on GCE node
I20260809 18:51:45.414224 26797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:45.414448 26797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:45.415582 26797 hybrid_clock.cc:648] HybridClock initialized: now 1786301505415570 us; error 29 us; skew 500 ppm
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:45.417930 26797 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:51:45.418314 26797 webserver.cc:460] Webserver started at http://127.23.218.195:38529/ using document root <none> and password file <none>
I20260809 18:51:45.418520 26797 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:45.418567 26797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:45.418682 26797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:45.419642 26797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "f32051a43b6b49569887817f7b429059"
format_stamp: "Formatted at 2026-08-09 18:51:45 on dist-test-slave-6zbq"
server_key: "181e9a8e48ec5a7b90f3e4f9952acd88"
server_key_iv: "67311250b470c11966009684550e95b0"
server_key_version: "encryptionkey@0"
I20260809 18:51:45.419970 26797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "f32051a43b6b49569887817f7b429059"
format_stamp: "Formatted at 2026-08-09 18:51:45 on dist-test-slave-6zbq"
server_key: "181e9a8e48ec5a7b90f3e4f9952acd88"
server_key_iv: "67311250b470c11966009684550e95b0"
server_key_version: "encryptionkey@0"
I20260809 18:51:45.421252 26797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:51:45.421901 26821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.422048 26797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:45.422138 26797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "f32051a43b6b49569887817f7b429059"
format_stamp: "Formatted at 2026-08-09 18:51:45 on dist-test-slave-6zbq"
server_key: "181e9a8e48ec5a7b90f3e4f9952acd88"
server_key_iv: "67311250b470c11966009684550e95b0"
server_key_version: "encryptionkey@0"
I20260809 18:51:45.422189 26797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:45.433696 26797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:45.435281 26797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:45.435380 26797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:45.435786 26797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:45.435827 26797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.435845 26797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:45.435856 26797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301505, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:45.442358 26797 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:42955
I20260809 18:51:45.442422 27012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:42955 every 8 connection(s)
I20260809 18:51:45.442720 26797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:51:45.445036 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.436180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:44129 (local address 127.23.218.254:34539)
0809 18:51:45.436244 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.436247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.436475 (+ 228us) server_negotiation.cc:407] Connection header received
0809 18:51:45.436928 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.436929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.436943 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.437019 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.437766 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.438543 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.439932 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.440237 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.440311 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.442636 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.442638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.442638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.442646 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.443846 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.444198 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.444199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.444199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.444257 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.444460 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.444461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.444462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.444571 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.444628 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.444868 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.444950 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 18:51:45.445169 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.436161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34539 (local address 127.23.218.195:44129)
0809 18:51:45.436381 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.436406 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.436815 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.437080 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.437084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.437403 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:45.437669 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.437675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.438635 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.438636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.439847 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.439851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.440319 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.440320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.440452 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.440940 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:45.440951 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:45.442536 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:45.443918 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.443925 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.443927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.444130 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.444310 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.444310 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.444311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.444407 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.444625 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:45.444632 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:45.444755 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.444838 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.444978 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":80,"threads_started":1}
I20260809 18:51:45.445576 27016 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34539
I20260809 18:51:45.445653 27016 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:45.445808 27016 heartbeater.cc:511] Master 127.23.218.254:34539 requested a full tablet report, sending...
I20260809 18:51:45.446180 26223 ts_manager.cc:194] Registered new tserver with Master: f32051a43b6b49569887817f7b429059 (127.23.218.195:42955)
I20260809 18:51:45.446642 26223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:44129
I20260809 18:51:45.450470 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 26797
I20260809 18:51:45.450549 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:51:45.450735 24427 external_mini_cluster.cc:1428] Setting key 3234b0a462c67051bad9ced3bf00e7a2
I20260809 18:51:45.451140 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.446502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34539 (local address 127.23.218.195:33067)
0809 18:51:45.446674 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.446684 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.446754 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.446867 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.446867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.447029 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:45.447081 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.447083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.447668 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.447668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.448157 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.448159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.448464 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.448465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.448538 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.448842 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:45.448846 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:45.449106 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:45.450166 (+ 1060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.450167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.450167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.450314 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.450494 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.450495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.450495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.450552 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.450853 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:45.450854 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:45.450895 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.450952 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.451042 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
I20260809 18:51:45.451140 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.446559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:33067 (local address 127.23.218.254:34539)
0809 18:51:45.446611 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.446613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.446731 (+ 118us) server_negotiation.cc:407] Connection header received
0809 18:51:45.446807 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.446808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.446817 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.446852 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.447146 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.447579 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.448225 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.448411 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.448461 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.449179 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.449180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.449181 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.449187 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.450081 (+ 894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.450392 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.450393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.450393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.450443 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.450631 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.450632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.450632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.450765 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.450838 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.450990 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.451052 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 18:51:45.451491 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301505, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301505, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:45.472793 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.467101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.23.218.254:34539)
0809 18:51:45.467146 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.467148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.467177 (+ 29us) server_negotiation.cc:407] Connection header received
0809 18:51:45.467235 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.467235 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.467244 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.467287 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.467791 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.468223 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.468879 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.469048 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.469106 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.470924 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.470925 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.470926 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.470932 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.471884 (+ 952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.472171 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.472172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.472172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.472218 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.472376 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.472377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.472377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.472464 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.472545 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.472651 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.472717 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:51:45.474030 26223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:51:45.475507 26223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:51:45.482946 27039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.479356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:46747 (local address 127.0.0.1:41284)
0809 18:51:45.479907 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.479915 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.479959 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.480107 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.480108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.480109 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:45.480150 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.480152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.480965 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.480966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.481927 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.481930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.482679 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.482680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.482754 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.482767 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.482835 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.482880 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":76,"threads_started":1}
I20260809 18:51:45.482982 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.479451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41284 (local address 127.23.218.194:46747)
0809 18:51:45.479727 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.479731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.479973 (+ 242us) server_negotiation.cc:407] Connection header received
0809 18:51:45.480019 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.480022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.480037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.480100 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:45.480216 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.480889 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.482001 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.482596 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.482673 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.482725 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.482852 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.482888 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":78,"threads_started":1}
I20260809 18:51:45.483438 27037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.477318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:42955 (local address 127.0.0.1:51924)
0809 18:51:45.479261 (+ 1943us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.479299 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.479358 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.479864 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.479867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.479869 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:45.479927 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.479928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.481174 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.481175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.482366 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.482369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.483213 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.483214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.483277 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.483281 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.483331 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.483368 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1915,"mutex_wait_us":38,"thread_start_us":1877,"threads_started":1}
I20260809 18:51:45.483458 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.479362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51924 (local address 127.23.218.195:42955)
0809 18:51:45.479498 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.479502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.479509 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:51:45.479536 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.479538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.479777 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.479842 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:45.480015 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.481065 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.482443 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.483136 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.483202 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.483252 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.483347 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.483380 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260809 18:51:45.483662 26677 tablet_service.cc:1467] Processing CreateTablet for tablet 5411d4814f4a4416bd694c550382a479 (DEFAULT_TABLE table=test-table [id=b9ef9fbf221548c580da3dfae5914c8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:45.483996 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.477321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40567 (local address 127.0.0.1:48610)
0809 18:51:45.479626 (+ 2305us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.479637 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.479694 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.479929 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.479929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.479931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:45.479968 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.479970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.481174 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.481175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.482411 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.482413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.483737 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.483739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.483797 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.483801 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.483895 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.483934 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2281,"mutex_wait_us":45,"thread_start_us":1903,"threads_started":1}
I20260809 18:51:45.484022 26677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5411d4814f4a4416bd694c550382a479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:45.484048 26915 tablet_service.cc:1467] Processing CreateTablet for tablet 5411d4814f4a4416bd694c550382a479 (DEFAULT_TABLE table=test-table [id=b9ef9fbf221548c580da3dfae5914c8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:45.484339 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.477420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48610 (local address 127.23.218.193:40567)
0809 18:51:45.479639 (+ 2219us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.479644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.479681 (+ 37us) server_negotiation.cc:407] Connection header received
0809 18:51:45.479787 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.479789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.479804 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.479870 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:45.480040 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.481085 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.482486 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.483392 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.483455 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.484165 (+ 710us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.484191 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.484227 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2182,"thread_start_us":1854,"threads_started":1}
I20260809 18:51:45.484407 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5411d4814f4a4416bd694c550382a479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:45.484917 26458 tablet_service.cc:1467] Processing CreateTablet for tablet 5411d4814f4a4416bd694c550382a479 (DEFAULT_TABLE table=test-table [id=b9ef9fbf221548c580da3dfae5914c8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:45.485265 26458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5411d4814f4a4416bd694c550382a479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:45.486915 27051 tablet_bootstrap.cc:492] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: Bootstrap starting.
I20260809 18:51:45.487504 27051 tablet_bootstrap.cc:654] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:45.487494 27052 tablet_bootstrap.cc:492] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: Bootstrap starting.
I20260809 18:51:45.487855 27051 log.cc:824] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:45.488405 27052 tablet_bootstrap.cc:654] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:45.488729 27052 log.cc:824] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:45.488816 27051 tablet_bootstrap.cc:492] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: No bootstrap required, opened a new log
I20260809 18:51:45.488901 27051 ts_tablet_manager.cc:1397] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:51:45.489499 27052 tablet_bootstrap.cc:492] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: No bootstrap required, opened a new log
I20260809 18:51:45.489560 27052 ts_tablet_manager.cc:1397] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:51:45.490499 27054 tablet_bootstrap.cc:492] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: Bootstrap starting.
I20260809 18:51:45.490444 27051 raft_consensus.cc:348] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } }
I20260809 18:51:45.490617 27051 raft_consensus.cc:374] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:45.490658 27051 raft_consensus.cc:729] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213496751f864fceb4bc29e000c3fa22, State: Initialized, Role: FOLLOWER
I20260809 18:51:45.490790 27051 consensus_queue.cc:260] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } }
I20260809 18:51:45.491041 27051 ts_tablet_manager.cc:1428] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:51:45.491080 26785 heartbeater.cc:503] Master 127.23.218.254:34539 was elected leader, sending a full tablet report...
I20260809 18:51:45.491118 27054 tablet_bootstrap.cc:654] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:45.491228 27052 raft_consensus.cc:348] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } }
I20260809 18:51:45.491370 27052 raft_consensus.cc:374] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:45.491374 27054 log.cc:824] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:45.491415 27052 raft_consensus.cc:729] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32051a43b6b49569887817f7b429059, State: Initialized, Role: FOLLOWER
I20260809 18:51:45.491524 27052 consensus_queue.cc:260] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } }
I20260809 18:51:45.491767 27052 ts_tablet_manager.cc:1428] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:51:45.491802 27016 heartbeater.cc:503] Master 127.23.218.254:34539 was elected leader, sending a full tablet report...
I20260809 18:51:45.492244 27054 tablet_bootstrap.cc:492] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: No bootstrap required, opened a new log
I20260809 18:51:45.492309 27054 ts_tablet_manager.cc:1397] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:51:45.493928 27054 raft_consensus.cc:348] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } }
I20260809 18:51:45.494045 27054 raft_consensus.cc:374] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:45.494064 27054 raft_consensus.cc:729] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5983bbff72c7496e8a3c85dd2758eb35, State: Initialized, Role: FOLLOWER
I20260809 18:51:45.494153 27054 consensus_queue.cc:260] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } }
I20260809 18:51:45.494378 27054 ts_tablet_manager.cc:1428] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:51:45.494417 26561 heartbeater.cc:503] Master 127.23.218.254:34539 was elected leader, sending a full tablet report...
W20260809 18:51:45.553511 26786 tablet.cc:2369] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:51:45.660048 26562 tablet.cc:2369] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:51:45.674774 27060 raft_consensus.cc:482] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:45.674858 27060 raft_consensus.cc:504] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } }
I20260809 18:51:45.675187 27060 leader_election.cc:288] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f32051a43b6b49569887817f7b429059 (127.23.218.195:42955), 213496751f864fceb4bc29e000c3fa22 (127.23.218.194:46747)
I20260809 18:51:45.678833 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.675416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:49355 (local address 127.23.218.195:42955)
0809 18:51:45.675471 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.675474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.675488 (+ 14us) server_negotiation.cc:407] Connection header received
0809 18:51:45.675556 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.675557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.675566 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.675626 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:45.675783 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.676516 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.677898 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.678477 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.678547 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.678556 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.678731 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.678766 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 18:51:45.678833 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.675456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:55101 (local address 127.23.218.194:46747)
0809 18:51:45.675493 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.675495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.675502 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:51:45.675556 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.675557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.675566 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.675626 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:45.675784 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.676533 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.677899 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.678477 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.678547 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.678556 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.678731 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.678768 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:51:45.678887 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.675327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:42955 (local address 127.23.218.193:49355)
0809 18:51:45.675437 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.675448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.675500 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.675638 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.675638 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.675642 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:45.675721 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.675722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.676587 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.676588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.677812 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.677815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.678555 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.678557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.678656 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.678662 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.678721 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.678778 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":15,"thread_start_us":54,"threads_started":1}
I20260809 18:51:45.678890 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.675328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:46747 (local address 127.23.218.193:55101)
0809 18:51:45.675437 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.675448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.675500 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.675638 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.675638 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.675641 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:45.675721 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.675722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.676607 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.676608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.677811 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.677815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.678555 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.678557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.678656 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.678663 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.678721 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.678777 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":71,"thread_start_us":46,"threads_started":1}
I20260809 18:51:45.679236 26946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5411d4814f4a4416bd694c550382a479" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32051a43b6b49569887817f7b429059" is_pre_election: true
I20260809 18:51:45.679281 26707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5411d4814f4a4416bd694c550382a479" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213496751f864fceb4bc29e000c3fa22" is_pre_election: true
I20260809 18:51:45.679394 26946 raft_consensus.cc:2454] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 0.
I20260809 18:51:45.679394 26707 raft_consensus.cc:2454] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 0.
I20260809 18:51:45.679617 26378 leader_election.cc:302] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213496751f864fceb4bc29e000c3fa22, 5983bbff72c7496e8a3c85dd2758eb35; no voters:
I20260809 18:51:45.679764 27060 raft_consensus.cc:2781] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:51:45.679823 27060 raft_consensus.cc:482] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:51:45.679847 27060 raft_consensus.cc:3037] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:45.680521 27060 raft_consensus.cc:504] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } }
I20260809 18:51:45.680644 27060 leader_election.cc:288] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 election: Requested vote from peers f32051a43b6b49569887817f7b429059 (127.23.218.195:42955), 213496751f864fceb4bc29e000c3fa22 (127.23.218.194:46747)
I20260809 18:51:45.680779 26707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5411d4814f4a4416bd694c550382a479" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213496751f864fceb4bc29e000c3fa22"
I20260809 18:51:45.680794 26946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5411d4814f4a4416bd694c550382a479" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32051a43b6b49569887817f7b429059"
I20260809 18:51:45.680860 26946 raft_consensus.cc:3037] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:45.680860 26707 raft_consensus.cc:3037] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:45.681715 26946 raft_consensus.cc:2454] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 1.
I20260809 18:51:45.681715 26707 raft_consensus.cc:2454] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 1.
I20260809 18:51:45.681862 26378 leader_election.cc:302] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213496751f864fceb4bc29e000c3fa22, 5983bbff72c7496e8a3c85dd2758eb35; no voters:
I20260809 18:51:45.681955 27060 raft_consensus.cc:2781] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:45.682063 27060 raft_consensus.cc:686] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 LEADER]: Becoming Leader. State: Replica: 5983bbff72c7496e8a3c85dd2758eb35, State: Running, Role: LEADER
I20260809 18:51:45.682152 27060 consensus_queue.cc:237] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } }
I20260809 18:51:45.682927 26223 catalog_manager.cc:5495] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5983bbff72c7496e8a3c85dd2758eb35 (127.23.218.193). New cstate: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } health_report { overall_health: UNKNOWN } } }
W20260809 18:51:45.693250 27017 tablet.cc:2369] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:51:45.708159 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.705827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.23.218.193:40567)
0809 18:51:45.705865 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.705867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.705875 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:51:45.705955 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.705956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.705972 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.706025 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:45.706161 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.706657 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.707419 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.707614 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.707668 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.707872 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:45.707962 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:45.708027 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.708078 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.708098 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:51:45.709576 26707 raft_consensus.cc:1261] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Refusing update from remote peer 5983bbff72c7496e8a3c85dd2758eb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:45.709604 26946 raft_consensus.cc:1261] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Refusing update from remote peer 5983bbff72c7496e8a3c85dd2758eb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:45.709883 27063 consensus_queue.cc:1035] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:45.709997 27060 consensus_queue.cc:1035] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:45.712707 27070 mvcc.cc:205] Tried to move back new op lower bound from 7316690967384510464 to 7316690967274774528. Current Snapshot: MvccSnapshot[applied={T|T < 7316690967384510464}]
I20260809 18:51:45.716135 26224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:45.716213 26224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:45.716902 26224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a: Sending DeleteTablet for 3 replicas of tablet 5411d4814f4a4416bd694c550382a479
I20260809 18:51:45.717254 26458 tablet_service.cc:1514] Processing DeleteTablet for tablet 5411d4814f4a4416bd694c550382a479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:45 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:48610
I20260809 18:51:45.717278 26915 tablet_service.cc:1514] Processing DeleteTablet for tablet 5411d4814f4a4416bd694c550382a479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:45 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:51924
I20260809 18:51:45.717283 26677 tablet_service.cc:1514] Processing DeleteTablet for tablet 5411d4814f4a4416bd694c550382a479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:45 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41284
I20260809 18:51:45.717465 27079 tablet_replica.cc:330] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: stopping tablet replica
I20260809 18:51:45.717554 27079 raft_consensus.cc:2229] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:51:45.717667 27079 raft_consensus.cc:2258] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:45.717674 26224 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:51:45.717808 26224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:51:45.718061 27082 tablet_replica.cc:330] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: stopping tablet replica
I20260809 18:51:45.718163 27082 raft_consensus.cc:2229] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:45.718277 27082 raft_consensus.cc:2258] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:45.719065 27079 ts_tablet_manager.cc:1905] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:45.719103 26458 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb0e980826e40c68706a3227ca610e0 (DEFAULT_TABLE table=test-table [id=c2a1ef866d8c4954b2dd455023050b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:45.719229 26458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb0e980826e40c68706a3227ca610e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:45.719353 26677 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb0e980826e40c68706a3227ca610e0 (DEFAULT_TABLE table=test-table [id=c2a1ef866d8c4954b2dd455023050b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:45.719461 26677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb0e980826e40c68706a3227ca610e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:45.720427 27080 tablet_replica.cc:330] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: stopping tablet replica
I20260809 18:51:45.720456 27054 tablet_bootstrap.cc:492] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: Bootstrap starting.
I20260809 18:51:45.720516 27080 raft_consensus.cc:2229] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:45.720619 27080 raft_consensus.cc:2258] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:45.720763 27051 tablet_bootstrap.cc:492] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: Bootstrap starting.
I20260809 18:51:45.720793 27079 ts_tablet_manager.cc:1918] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:51:45.720856 27079 log.cc:1196] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/5411d4814f4a4416bd694c550382a479
I20260809 18:51:45.720877 27054 tablet_bootstrap.cc:654] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:45.721107 27079 ts_tablet_manager.cc:1939] T 5411d4814f4a4416bd694c550382a479 P 5983bbff72c7496e8a3c85dd2758eb35: Deleting consensus metadata
I20260809 18:51:45.721155 27051 tablet_bootstrap.cc:654] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:45.721230 26915 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb0e980826e40c68706a3227ca610e0 (DEFAULT_TABLE table=test-table [id=c2a1ef866d8c4954b2dd455023050b0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:45.721334 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cb0e980826e40c68706a3227ca610e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:45.721680 27051 tablet_bootstrap.cc:492] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: No bootstrap required, opened a new log
I20260809 18:51:45.721721 27051 ts_tablet_manager.cc:1397] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:45.721753 27080 ts_tablet_manager.cc:1905] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:45.721860 27051 raft_consensus.cc:348] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.721920 27051 raft_consensus.cc:374] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:45.721937 27051 raft_consensus.cc:729] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213496751f864fceb4bc29e000c3fa22, State: Initialized, Role: FOLLOWER
I20260809 18:51:45.721977 27051 consensus_queue.cc:260] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.722064 27051 ts_tablet_manager.cc:1428] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.722236 27082 ts_tablet_manager.cc:1905] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:45.722591 27052 tablet_bootstrap.cc:492] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: Bootstrap starting.
I20260809 18:51:45.722977 27052 tablet_bootstrap.cc:654] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:45.723037 26194 catalog_manager.cc:4841] TS 5983bbff72c7496e8a3c85dd2758eb35 (127.23.218.193:40567): tablet 5411d4814f4a4416bd694c550382a479 (table test-table [id=b9ef9fbf221548c580da3dfae5914c8a]) successfully deleted
I20260809 18:51:45.723088 27080 ts_tablet_manager.cc:1918] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:51:45.723145 27080 log.cc:1196] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/5411d4814f4a4416bd694c550382a479
I20260809 18:51:45.723412 27080 ts_tablet_manager.cc:1939] T 5411d4814f4a4416bd694c550382a479 P f32051a43b6b49569887817f7b429059: Deleting consensus metadata
I20260809 18:51:45.723527 27054 tablet_bootstrap.cc:492] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: No bootstrap required, opened a new log
I20260809 18:51:45.723568 27054 ts_tablet_manager.cc:1397] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 18:51:45.723618 27082 ts_tablet_manager.cc:1918] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:51:45.723651 27052 tablet_bootstrap.cc:492] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: No bootstrap required, opened a new log
I20260809 18:51:45.723682 27082 log.cc:1196] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/5411d4814f4a4416bd694c550382a479
I20260809 18:51:45.723687 27052 ts_tablet_manager.cc:1397] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:45.723699 27054 raft_consensus.cc:348] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.723757 27054 raft_consensus.cc:374] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:45.723774 27054 raft_consensus.cc:729] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5983bbff72c7496e8a3c85dd2758eb35, State: Initialized, Role: FOLLOWER
I20260809 18:51:45.723788 26198 catalog_manager.cc:4841] TS f32051a43b6b49569887817f7b429059 (127.23.218.195:42955): tablet 5411d4814f4a4416bd694c550382a479 (table test-table [id=b9ef9fbf221548c580da3dfae5914c8a]) successfully deleted
I20260809 18:51:45.723814 27054 consensus_queue.cc:260] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.723829 27052 raft_consensus.cc:348] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.723891 27054 ts_tablet_manager.cc:1428] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.723909 27052 raft_consensus.cc:374] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:45.723910 27082 ts_tablet_manager.cc:1939] T 5411d4814f4a4416bd694c550382a479 P 213496751f864fceb4bc29e000c3fa22: Deleting consensus metadata
I20260809 18:51:45.723932 27052 raft_consensus.cc:729] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32051a43b6b49569887817f7b429059, State: Initialized, Role: FOLLOWER
I20260809 18:51:45.723971 27052 consensus_queue.cc:260] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.724054 27052 ts_tablet_manager.cc:1428] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.724335 26194 catalog_manager.cc:4841] TS 213496751f864fceb4bc29e000c3fa22 (127.23.218.194:46747): tablet 5411d4814f4a4416bd694c550382a479 (table test-table [id=b9ef9fbf221548c580da3dfae5914c8a]) successfully deleted
I20260809 18:51:45.779495 27060 raft_consensus.cc:482] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:45.779572 27060 raft_consensus.cc:504] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.779713 27060 leader_election.cc:288] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213496751f864fceb4bc29e000c3fa22 (127.23.218.194:46747), f32051a43b6b49569887817f7b429059 (127.23.218.195:42955)
I20260809 18:51:45.779857 26946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb0e980826e40c68706a3227ca610e0" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32051a43b6b49569887817f7b429059" is_pre_election: true
I20260809 18:51:45.779857 26707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb0e980826e40c68706a3227ca610e0" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213496751f864fceb4bc29e000c3fa22" is_pre_election: true
I20260809 18:51:45.779942 26946 raft_consensus.cc:2454] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 0.
I20260809 18:51:45.779953 26707 raft_consensus.cc:2454] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 0.
I20260809 18:51:45.780072 26378 leader_election.cc:302] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 213496751f864fceb4bc29e000c3fa22, 5983bbff72c7496e8a3c85dd2758eb35, f32051a43b6b49569887817f7b429059; no voters:
I20260809 18:51:45.780193 27060 raft_consensus.cc:2781] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:51:45.780241 27060 raft_consensus.cc:482] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:51:45.780261 27060 raft_consensus.cc:3037] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:45.780989 27060 raft_consensus.cc:504] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.781129 27060 leader_election.cc:288] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 election: Requested vote from peers 213496751f864fceb4bc29e000c3fa22 (127.23.218.194:46747), f32051a43b6b49569887817f7b429059 (127.23.218.195:42955)
I20260809 18:51:45.781252 26707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb0e980826e40c68706a3227ca610e0" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213496751f864fceb4bc29e000c3fa22"
I20260809 18:51:45.781267 26946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb0e980826e40c68706a3227ca610e0" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32051a43b6b49569887817f7b429059"
I20260809 18:51:45.781323 26707 raft_consensus.cc:3037] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:45.781323 26946 raft_consensus.cc:3037] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:45.781980 26946 raft_consensus.cc:2454] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 1.
I20260809 18:51:45.781980 26707 raft_consensus.cc:2454] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 1.
I20260809 18:51:45.782148 26383 leader_election.cc:302] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5983bbff72c7496e8a3c85dd2758eb35, f32051a43b6b49569887817f7b429059; no voters:
I20260809 18:51:45.782238 27060 raft_consensus.cc:2781] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:45.782279 27060 raft_consensus.cc:686] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 LEADER]: Becoming Leader. State: Replica: 5983bbff72c7496e8a3c85dd2758eb35, State: Running, Role: LEADER
I20260809 18:51:45.782318 27060 consensus_queue.cc:237] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.782814 26224 catalog_manager.cc:5495] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5983bbff72c7496e8a3c85dd2758eb35 (127.23.218.193). New cstate: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } health_report { overall_health: UNKNOWN } } }
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301504, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:45.825918 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.819001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.23.218.254:34539)
0809 18:51:45.819059 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.819062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.819192 (+ 130us) server_negotiation.cc:407] Connection header received
0809 18:51:45.819268 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.819269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.819277 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.819323 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.819890 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.820359 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.821373 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.821540 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.821579 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.823566 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.823567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.823568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.823575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.824823 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.825156 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.825158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.825158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.825208 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.825389 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.825389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.825390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.825510 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.825565 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.825702 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.825801 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":31}
I20260809 18:51:45.825971 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.818937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34539 (local address 127.0.0.1:33628)
0809 18:51:45.819107 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.819119 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.819208 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.819345 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.819347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.819764 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:45.819821 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.819825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.820434 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.820436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.821305 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.821307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.821588 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.821589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.821655 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.822108 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:45.822113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:45.823490 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:45.824895 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.824902 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.824904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.825075 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.825282 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:45.825282 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:45.825283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:45.825339 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:45.825564 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:45.825571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:45.825618 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.825676 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.825854 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":54,"thread_start_us":83,"threads_started":1}
I20260809 18:51:45.827639 26224 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:33628:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 18:51:45.827916 26224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:51:45.829196 26458 tablet_service.cc:1467] Processing CreateTablet for tablet a636c9020bc24d7ba99c58a91c0af156 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=716bf85d6c2d4a568e8dba6023f802ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 18:51:45.829185 26915 tablet_service.cc:1467] Processing CreateTablet for tablet a636c9020bc24d7ba99c58a91c0af156 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=716bf85d6c2d4a568e8dba6023f802ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 18:51:45.829236 26677 tablet_service.cc:1467] Processing CreateTablet for tablet a636c9020bc24d7ba99c58a91c0af156 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=716bf85d6c2d4a568e8dba6023f802ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 18:51:45.829336 26458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a636c9020bc24d7ba99c58a91c0af156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:45.829339 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a636c9020bc24d7ba99c58a91c0af156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:45.829349 26677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a636c9020bc24d7ba99c58a91c0af156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:45.830396 27054 tablet_bootstrap.cc:492] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: Bootstrap starting.
I20260809 18:51:45.830660 27054 tablet_bootstrap.cc:654] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:45.830792 27052 tablet_bootstrap.cc:492] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: Bootstrap starting.
I20260809 18:51:45.830802 27051 tablet_bootstrap.cc:492] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: Bootstrap starting.
I20260809 18:51:45.831190 27051 tablet_bootstrap.cc:654] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:45.831223 27052 tablet_bootstrap.cc:654] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:45.831359 27054 tablet_bootstrap.cc:492] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: No bootstrap required, opened a new log
I20260809 18:51:45.831400 27054 ts_tablet_manager.cc:1397] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 18:51:45.831507 27054 raft_consensus.cc:348] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.831576 27054 raft_consensus.cc:374] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:45.831595 27054 raft_consensus.cc:729] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5983bbff72c7496e8a3c85dd2758eb35, State: Initialized, Role: FOLLOWER
I20260809 18:51:45.831650 27054 consensus_queue.cc:260] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.831743 27054 ts_tablet_manager.cc:1428] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.831787 27060 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.831883 27060 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:45.832020 27051 tablet_bootstrap.cc:492] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: No bootstrap required, opened a new log
I20260809 18:51:45.832055 27051 ts_tablet_manager.cc:1397] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:45.832196 27051 raft_consensus.cc:348] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.832266 27051 raft_consensus.cc:374] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:45.832265 27052 tablet_bootstrap.cc:492] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: No bootstrap required, opened a new log
I20260809 18:51:45.832284 27051 raft_consensus.cc:729] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213496751f864fceb4bc29e000c3fa22, State: Initialized, Role: FOLLOWER
I20260809 18:51:45.832304 27052 ts_tablet_manager.cc:1397] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:51:45.832324 27051 consensus_queue.cc:260] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.832405 27051 ts_tablet_manager.cc:1428] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.832458 27052 raft_consensus.cc:348] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.832450 27056 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.832516 27052 raft_consensus.cc:374] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:45.832535 27052 raft_consensus.cc:729] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32051a43b6b49569887817f7b429059, State: Initialized, Role: FOLLOWER
I20260809 18:51:45.832527 27056 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:45.832571 27052 consensus_queue.cc:260] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.832662 27052 ts_tablet_manager.cc:1428] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:45.832789 27058 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.832868 27058 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:45.884557 27056 raft_consensus.cc:482] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:45.884627 27056 raft_consensus.cc:504] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.884898 27056 leader_election.cc:288] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5983bbff72c7496e8a3c85dd2758eb35 (127.23.218.193:40567), f32051a43b6b49569887817f7b429059 (127.23.218.195:42955)
I20260809 18:51:45.884992 27060 raft_consensus.cc:482] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:45.885044 27060 raft_consensus.cc:504] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.885151 27060 leader_election.cc:288] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213496751f864fceb4bc29e000c3fa22 (127.23.218.194:46747), f32051a43b6b49569887817f7b429059 (127.23.218.195:42955)
I20260809 18:51:45.885341 26707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a636c9020bc24d7ba99c58a91c0af156" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213496751f864fceb4bc29e000c3fa22" is_pre_election: true
I20260809 18:51:45.885334 26946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a636c9020bc24d7ba99c58a91c0af156" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32051a43b6b49569887817f7b429059" is_pre_election: true
I20260809 18:51:45.885401 26707 raft_consensus.cc:2454] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 0.
I20260809 18:51:45.885401 26946 raft_consensus.cc:2454] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 0.
I20260809 18:51:45.885540 26383 leader_election.cc:302] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5983bbff72c7496e8a3c85dd2758eb35, f32051a43b6b49569887817f7b429059; no voters:
I20260809 18:51:45.885643 27060 raft_consensus.cc:2781] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:51:45.885680 27060 raft_consensus.cc:482] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:51:45.885699 27060 raft_consensus.cc:3037] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:45.886368 27060 raft_consensus.cc:504] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.886482 27060 leader_election.cc:288] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 election: Requested vote from peers 213496751f864fceb4bc29e000c3fa22 (127.23.218.194:46747), f32051a43b6b49569887817f7b429059 (127.23.218.195:42955)
I20260809 18:51:45.886620 26946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a636c9020bc24d7ba99c58a91c0af156" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32051a43b6b49569887817f7b429059"
I20260809 18:51:45.886682 26946 raft_consensus.cc:3037] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:45.886965 26707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a636c9020bc24d7ba99c58a91c0af156" candidate_uuid: "5983bbff72c7496e8a3c85dd2758eb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213496751f864fceb4bc29e000c3fa22"
I20260809 18:51:45.887019 26707 raft_consensus.cc:3037] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:45.887346 26946 raft_consensus.cc:2454] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 1.
I20260809 18:51:45.887490 26383 leader_election.cc:302] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5983bbff72c7496e8a3c85dd2758eb35, f32051a43b6b49569887817f7b429059; no voters:
I20260809 18:51:45.887606 27060 raft_consensus.cc:2781] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:45.887633 26707 raft_consensus.cc:2454] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983bbff72c7496e8a3c85dd2758eb35 in term 1.
I20260809 18:51:45.887650 27060 raft_consensus.cc:686] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 LEADER]: Becoming Leader. State: Replica: 5983bbff72c7496e8a3c85dd2758eb35, State: Running, Role: LEADER
I20260809 18:51:45.887686 27060 consensus_queue.cc:237] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } }
I20260809 18:51:45.887912 27063 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: TxnStatusTablet state changed. Reason: New leader 5983bbff72c7496e8a3c85dd2758eb35. Latest consensus state: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.887972 27063 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: This TxnStatusTablet replica's current role is: LEADER
I20260809 18:51:45.888360 27108 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 18:51:45.888341 26224 catalog_manager.cc:5495] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5983bbff72c7496e8a3c85dd2758eb35 (127.23.218.193). New cstate: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } health_report { overall_health: UNKNOWN } } }
I20260809 18:51:45.888460 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.885170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:51631 (local address 127.23.218.195:42955)
0809 18:51:45.885220 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.885222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.885321 (+ 99us) server_negotiation.cc:407] Connection header received
0809 18:51:45.885406 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.885407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.885415 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.885442 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:45.885708 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.886408 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.887612 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.888134 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.888172 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.888180 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.888371 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.888409 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 18:51:45.888413 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.885139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:39339 (local address 127.23.218.193:40567)
0809 18:51:45.885235 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.885237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.885299 (+ 62us) server_negotiation.cc:407] Connection header received
0809 18:51:45.885403 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.885403 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.885414 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.885442 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:45.885664 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.886114 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.887477 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.887999 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.888041 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.888049 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.888283 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.888319 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 18:51:45.888667 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.885021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40567 (local address 127.23.218.194:39339)
0809 18:51:45.885242 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.885252 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.885326 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.885533 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.885534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.885536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:45.885584 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.885586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.886195 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.886197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.887381 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.887384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.888126 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.888127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.888200 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.888204 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.888381 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.888478 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":54,"threads_started":1}
I20260809 18:51:45.888847 27103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.885021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:42955 (local address 127.23.218.194:51631)
0809 18:51:45.885271 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.885279 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.885341 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.885502 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.885503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.885505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:45.885597 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.885598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.886485 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.886486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.887529 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.887531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.888230 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.888231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.888298 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.888301 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.888737 (+ 436us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.888777 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":8,"thread_start_us":45,"threads_started":1}
I20260809 18:51:45.888971 26491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a636c9020bc24d7ba99c58a91c0af156" candidate_uuid: "213496751f864fceb4bc29e000c3fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5983bbff72c7496e8a3c85dd2758eb35" is_pre_election: true
I20260809 18:51:45.889147 26946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a636c9020bc24d7ba99c58a91c0af156" candidate_uuid: "213496751f864fceb4bc29e000c3fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f32051a43b6b49569887817f7b429059" is_pre_election: true
I20260809 18:51:45.889221 26946 raft_consensus.cc:2379] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 213496751f864fceb4bc29e000c3fa22 in current term 1: Already voted for candidate 5983bbff72c7496e8a3c85dd2758eb35 in this term.
I20260809 18:51:45.889344 26603 leader_election.cc:302] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213496751f864fceb4bc29e000c3fa22; no voters: 5983bbff72c7496e8a3c85dd2758eb35, f32051a43b6b49569887817f7b429059
I20260809 18:51:45.889452 27056 raft_consensus.cc:2726] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 18:51:45.889901 27108 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 18:51:45.906083 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.903705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45750 (local address 127.23.218.193:40567)
0809 18:51:45.903759 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.903761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.903769 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:51:45.903789 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.903790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.903799 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.903853 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:45.903959 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.904651 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.905361 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.905650 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.905731 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.905868 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:45.905898 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:45.905944 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.905984 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.906000 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:51:45.906103 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.903521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40567 (local address 127.0.0.1:45750)
0809 18:51:45.903588 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.903606 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.903643 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.903863 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.903864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.903865 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:51:45.903908 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.903910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.904739 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.904740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.905290 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.905292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.905718 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.905718 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.905778 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.905796 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:51:45.905947 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:51:45.905951 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.905979 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.906030 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 18:51:45.907003 26946 raft_consensus.cc:1261] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Refusing update from remote peer 5983bbff72c7496e8a3c85dd2758eb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:45.907133 26707 raft_consensus.cc:1261] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Refusing update from remote peer 5983bbff72c7496e8a3c85dd2758eb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:45.907226 27063 consensus_queue.cc:1035] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:45.907291 27060 consensus_queue.cc:1035] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:45.907967 27058 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: TxnStatusTablet state changed. Reason: New leader 5983bbff72c7496e8a3c85dd2758eb35. Latest consensus state: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.908061 27058 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:45.908041 27056 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: TxnStatusTablet state changed. Reason: New leader 5983bbff72c7496e8a3c85dd2758eb35. Latest consensus state: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.908145 27056 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:45.908385 27063 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.908450 27063 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: This TxnStatusTablet replica's current role is: LEADER
I20260809 18:51:45.908658 27060 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.908727 27060 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P 5983bbff72c7496e8a3c85dd2758eb35: This TxnStatusTablet replica's current role is: LEADER
I20260809 18:51:45.908912 27066 mvcc.cc:205] Tried to move back new op lower bound from 7316690968193679360 to 7316690968116199424. Current Snapshot: MvccSnapshot[applied={T|T < 7316690968193679360 or (T in {7316690968193679360})}]
I20260809 18:51:45.909191 27058 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.909272 27058 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P f32051a43b6b49569887817f7b429059: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:45.909582 27071 mvcc.cc:205] Tried to move back new op lower bound from 7316690968193679360 to 7316690968116199424. Current Snapshot: MvccSnapshot[applied={T|T < 7316690968193679360 or (T in {7316690968193679360})}]
I20260809 18:51:45.910014 27056 tablet_replica.cc:439] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5983bbff72c7496e8a3c85dd2758eb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983bbff72c7496e8a3c85dd2758eb35" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40567 } } peers { permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 } } peers { permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 } } }
I20260809 18:51:45.910090 27056 tablet_replica.cc:442] T a636c9020bc24d7ba99c58a91c0af156 P 213496751f864fceb4bc29e000c3fa22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:45.914470 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.912101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:41297 (local address 127.23.218.193:40567)
0809 18:51:45.912189 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.912191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.912253 (+ 62us) server_negotiation.cc:407] Connection header received
0809 18:51:45.912295 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.912296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.912305 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.912359 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:45.912428 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.913093 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.913759 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.914046 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.914111 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.914234 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:45.914262 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:45.914303 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.914365 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.914382 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 18:51:45.914484 27116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.912055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40567 (local address 127.23.218.193:41297)
0809 18:51:45.912205 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:45.912213 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:45.912260 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:45.912362 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:45.912362 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:45.912363 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:51:45.912395 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.912396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.913166 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.913166 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.913695 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:45.913696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.914108 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.914108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:45.914150 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.914162 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:51:45.914312 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:51:45.914313 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 18:51:45.914377 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 18:51:45.914416 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260809 18:51:45.916361 26707 raft_consensus.cc:1261] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Refusing update from remote peer 5983bbff72c7496e8a3c85dd2758eb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:45.916357 26946 raft_consensus.cc:1261] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Refusing update from remote peer 5983bbff72c7496e8a3c85dd2758eb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:45.916535 27063 consensus_queue.cc:1035] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213496751f864fceb4bc29e000c3fa22" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:45.916625 27060 consensus_queue.cc:1035] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f32051a43b6b49569887817f7b429059" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 42955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:45.926820 26223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:45.926890 26223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:45.927412 26223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d0565e5282e4022aa383150d02d277a: Sending DeleteTablet for 3 replicas of tablet 9cb0e980826e40c68706a3227ca610e0
I20260809 18:51:45.927649 26915 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb0e980826e40c68706a3227ca610e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:45 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:51924
I20260809 18:51:45.927735 27080 tablet_replica.cc:330] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: stopping tablet replica
I20260809 18:51:45.927780 27080 raft_consensus.cc:2229] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:45.927810 27080 raft_consensus.cc:2258] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:45.927870 26458 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb0e980826e40c68706a3227ca610e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:45 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:48610
I20260809 18:51:45.927934 27079 tablet_replica.cc:330] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: stopping tablet replica
I20260809 18:51:45.927968 27079 raft_consensus.cc:2229] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:51:45.928009 27079 raft_consensus.cc:2258] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:45.928332 27080 ts_tablet_manager.cc:1905] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:45.929744 27080 ts_tablet_manager.cc:1918] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:51:45.929790 27080 log.cc:1196] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/9cb0e980826e40c68706a3227ca610e0
I20260809 18:51:45.929976 27080 ts_tablet_manager.cc:1939] T 9cb0e980826e40c68706a3227ca610e0 P f32051a43b6b49569887817f7b429059: Deleting consensus metadata
I20260809 18:51:45.930269 26198 catalog_manager.cc:4841] TS f32051a43b6b49569887817f7b429059 (127.23.218.195:42955): tablet 9cb0e980826e40c68706a3227ca610e0 (table test-table [id=c2a1ef866d8c4954b2dd455023050b0b]) successfully deleted
I20260809 18:51:45.930331 27079 ts_tablet_manager.cc:1905] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:45.931746 27079 ts_tablet_manager.cc:1918] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:51:45.931789 27079 log.cc:1196] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/9cb0e980826e40c68706a3227ca610e0
I20260809 18:51:45.931972 27079 ts_tablet_manager.cc:1939] T 9cb0e980826e40c68706a3227ca610e0 P 5983bbff72c7496e8a3c85dd2758eb35: Deleting consensus metadata
I20260809 18:51:45.932403 26677 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cb0e980826e40c68706a3227ca610e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:45 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41284
I20260809 18:51:45.932507 27082 tablet_replica.cc:330] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: stopping tablet replica
I20260809 18:51:45.932569 27082 raft_consensus.cc:2229] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:45.932602 27082 raft_consensus.cc:2258] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:45.932598 26194 catalog_manager.cc:4841] TS 5983bbff72c7496e8a3c85dd2758eb35 (127.23.218.193:40567): tablet 9cb0e980826e40c68706a3227ca610e0 (table test-table [id=c2a1ef866d8c4954b2dd455023050b0b]) successfully deleted
I20260809 18:51:45.932893 27082 ts_tablet_manager.cc:1905] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:45.934258 27082 ts_tablet_manager.cc:1918] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:51:45.934319 27082 log.cc:1196] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/9cb0e980826e40c68706a3227ca610e0
Aug 09 18:51:45 dist-test-slave-6zbq krb5kdc[26138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301505, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:51:45.934525 27082 ts_tablet_manager.cc:1939] T 9cb0e980826e40c68706a3227ca610e0 P 213496751f864fceb4bc29e000c3fa22: Deleting consensus metadata
I20260809 18:51:45.934795 26194 catalog_manager.cc:4841] TS 213496751f864fceb4bc29e000c3fa22 (127.23.218.194:46747): tablet 9cb0e980826e40c68706a3227ca610e0 (table test-table [id=c2a1ef866d8c4954b2dd455023050b0b]) successfully deleted
I20260809 18:51:45.937280 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.928271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45758 (local address 127.23.218.193:40567)
0809 18:51:45.930089 (+ 1818us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.930091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.930100 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:51:45.930118 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.930119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.930128 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.930201 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.930823 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.931238 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.931909 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.932100 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.932150 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.934941 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.934949 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.934950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.934956 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.936293 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.936597 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.936601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.936601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.936651 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.936818 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.936819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.936819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.936938 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.937059 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.937129 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.937202 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1795}
W20260809 18:51:45.937502 26548 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45758
I20260809 18:51:45.944157 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:45.938329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33642 (local address 127.23.218.254:34539)
0809 18:51:45.938398 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:45.938400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:45.938425 (+ 25us) server_negotiation.cc:407] Connection header received
0809 18:51:45.938475 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:45.938475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:45.938484 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:45.938543 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:45.938958 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.939684 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.940380 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:45.940698 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:45.940760 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:45.942047 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:45.942048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:45.942049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:45.942055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:45.943227 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.943556 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.943557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.943557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.943614 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:45.943788 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:45.943788 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:45.943789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:45.943879 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:45.943942 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:45.944051 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 18:51:45.944104 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 18:51:45.944341 26223 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33642
I20260809 18:51:45.944967 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 26352
I20260809 18:51:45.959195 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 26573
I20260809 18:51:45.967214 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 26797
I20260809 18:51:45.977679 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 26164
2026-08-09T18:51:45Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2371 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:51:46 dist-test-slave-6zbq krb5kdc[27132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:51:46 dist-test-slave-6zbq krb5kdc[27132](info): set up 2 sockets
Aug 09 18:51:46 dist-test-slave-6zbq krb5kdc[27132](info): commencing operation
krb5kdc: starting...
W20260809 18:51:47.104943 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:51:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:51:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:51:47.206975 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:39039
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:39039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:47.283785 27154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:47.283968 27154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:47.283999 27154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:47.285106 27154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:51:47.285144 27154 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 18:51:47.285163 27154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:47.285180 27154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:47.285194 27154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:51:47.285210 27154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:51:47.286643 27154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:39039
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:39039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:47.286890 27154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:47.287081 27154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:47.291440 27165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:47.291445 27168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:47.291445 27163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:47.291640 27154 server_base.cc:1034] running on GCE node
I20260809 18:51:47.291913 27154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:47.292199 27154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:47.293334 27154 hybrid_clock.cc:648] HybridClock initialized: now 1786301507293321 us; error 28 us; skew 500 ppm
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:47.295536 27154 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:51:47.295904 27154 webserver.cc:460] Webserver started at http://127.23.218.254:44089/ using document root <none> and password file <none>
I20260809 18:51:47.296103 27154 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:47.296157 27154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:47.296277 27154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:47.297219 27154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "f0857a726cc84d02820b23ddb3d8cf76"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "ba22e7923c39f387fa9264ce63623a5e"
server_key_iv: "cdb1b30213536aab8663aa8c007d55da"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.297531 27154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "f0857a726cc84d02820b23ddb3d8cf76"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "ba22e7923c39f387fa9264ce63623a5e"
server_key_iv: "cdb1b30213536aab8663aa8c007d55da"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.298729 27154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:51:47.299340 27184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.299479 27154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:47.299547 27154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "f0857a726cc84d02820b23ddb3d8cf76"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "ba22e7923c39f387fa9264ce63623a5e"
server_key_iv: "cdb1b30213536aab8663aa8c007d55da"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.299606 27154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:47.327133 27154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:47.328715 27154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:47.328842 27154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:47.333135 27154 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:39039
I20260809 18:51:47.333173 27287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:39039 every 8 connection(s)
I20260809 18:51:47.333460 27154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:51:47.334165 27288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:47.336349 27288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76: Bootstrap starting.
I20260809 18:51:47.336889 27288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:47.337136 27288 log.cc:824] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:47.337790 27288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76: No bootstrap required, opened a new log
I20260809 18:51:47.340499 27288 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857a726cc84d02820b23ddb3d8cf76" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39039 } }
I20260809 18:51:47.340672 27288 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:47.340718 27288 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0857a726cc84d02820b23ddb3d8cf76, State: Initialized, Role: FOLLOWER
I20260809 18:51:47.340853 27288 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857a726cc84d02820b23ddb3d8cf76" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39039 } }
I20260809 18:51:47.340917 27288 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:51:47.340948 27288 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:51:47.340993 27288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:47.341027 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27154
I20260809 18:51:47.341133 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:51:47.341288 24427 external_mini_cluster.cc:1428] Setting key 9008cdb81613d9add0b84ee449481074
I20260809 18:51:47.341611 27288 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857a726cc84d02820b23ddb3d8cf76" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39039 } }
I20260809 18:51:47.341737 27288 leader_election.cc:302] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0857a726cc84d02820b23ddb3d8cf76; no voters:
I20260809 18:51:47.341897 27288 leader_election.cc:288] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:51:47.341933 27293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:47.342096 27293 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [term 1 LEADER]: Becoming Leader. State: Replica: f0857a726cc84d02820b23ddb3d8cf76, State: Running, Role: LEADER
I20260809 18:51:47.342193 27288 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:51:47.342222 27293 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857a726cc84d02820b23ddb3d8cf76" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39039 } }
I20260809 18:51:47.342623 27293 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0857a726cc84d02820b23ddb3d8cf76. Latest consensus state: current_term: 1 leader_uuid: "f0857a726cc84d02820b23ddb3d8cf76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857a726cc84d02820b23ddb3d8cf76" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39039 } } }
I20260809 18:51:47.342764 27293 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:47.343201 27309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:51:47.343586 27295 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0857a726cc84d02820b23ddb3d8cf76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857a726cc84d02820b23ddb3d8cf76" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39039 } } }
I20260809 18:51:47.343940 27295 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76 [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:47.344080 27309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:51:47.345633 27309 catalog_manager.cc:1347] Generated new cluster ID: 7c2af6184cc243a599187b5d2d2944c5
I20260809 18:51:47.345685 27309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:47.351923 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.341824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.23.218.254:39039)
0809 18:51:47.341976 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.341985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.342018 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:51:47.342131 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.342138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.342203 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.342372 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:47.343080 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.344390 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.345306 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.345714 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.345805 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.347977 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:47.347981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:47.347981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:47.347991 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:47.350522 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.350834 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.350837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.350838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.350927 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.351097 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.351098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.351098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.351326 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:47.351470 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.351563 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.351775 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":56,"threads_started":1}
I20260809 18:51:47.369843 27309 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:51:47.370299 27309 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:51:47.374150 27309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76: Generated new TSK 0
I20260809 18:51:47.374300 27309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:51:47.402829 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:39039
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:47.490125 27336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:47.490303 27336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:47.490326 27336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:47.491514 27336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:47.491559 27336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:47.491590 27336 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 18:51:47.491612 27336 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 18:51:47.491636 27336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:51:47.493157 27336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.218.254:39039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:47.493372 27336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:47.493573 27336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:47.498179 27343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:47.498167 27347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:47.498251 27342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:47.498359 27336 server_base.cc:1034] running on GCE node
I20260809 18:51:47.498535 27336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:47.498735 27336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:47.499866 27336 hybrid_clock.cc:648] HybridClock initialized: now 1786301507499851 us; error 28 us; skew 500 ppm
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:47.502257 27336 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:51:47.502698 27336 webserver.cc:460] Webserver started at http://127.23.218.193:36363/ using document root <none> and password file <none>
I20260809 18:51:47.502907 27336 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:47.502954 27336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:47.503077 27336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:47.504082 27336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "d8e728dc64844133b2b732a7b36a991a"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "4dc55f93d89768d95158824f747e3d9f"
server_key_iv: "9199f58b1be23e33d20090aacd53beac"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.504419 27336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "d8e728dc64844133b2b732a7b36a991a"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "4dc55f93d89768d95158824f747e3d9f"
server_key_iv: "9199f58b1be23e33d20090aacd53beac"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.505621 27336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:51:47.506220 27358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.506412 27336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:47.506495 27336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "d8e728dc64844133b2b732a7b36a991a"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "4dc55f93d89768d95158824f747e3d9f"
server_key_iv: "9199f58b1be23e33d20090aacd53beac"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.506561 27336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:47.522336 27336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:47.524333 27336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:47.524458 27336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:47.524915 27336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:47.524957 27336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.524986 27336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:47.525010 27336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:47.531934 27336 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:41519
I20260809 18:51:47.532002 27551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:41519 every 8 connection(s)
I20260809 18:51:47.532303 27336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:51:47.534600 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.525336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:53563 (local address 127.23.218.254:39039)
0809 18:51:47.525398 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.525400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.525621 (+ 221us) server_negotiation.cc:407] Connection header received
0809 18:51:47.526259 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.526260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.526276 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.526377 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:47.527234 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.527790 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.529252 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.529561 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.529636 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.532129 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:47.532130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:47.532131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:47.532138 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:47.533424 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.533802 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.533802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.533803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.533847 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.534007 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.534007 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.534008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.534111 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:47.534161 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.534477 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.534540 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 18:51:47.534806 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.525312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:39039 (local address 127.23.218.193:53563)
0809 18:51:47.525485 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.525535 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.526144 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.526486 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.526491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.526830 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:47.527134 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.527141 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.527864 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.527865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.529140 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.529143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.529663 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.529663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.529816 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.530325 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:47.530335 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:47.532026 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:47.533508 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.533514 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.533516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.533736 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.533884 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.533884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.533885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.533968 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.534187 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:47.534198 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:47.534313 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.534459 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.534586 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260809 18:51:47.535266 27553 heartbeater.cc:347] Connected to a master server at 127.23.218.254:39039
I20260809 18:51:47.535342 27553 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:47.535503 27553 heartbeater.cc:511] Master 127.23.218.254:39039 requested a full tablet report, sending...
I20260809 18:51:47.536051 27208 ts_manager.cc:194] Registered new tserver with Master: d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519)
I20260809 18:51:47.536907 27208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:53563
I20260809 18:51:47.537530 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27336
I20260809 18:51:47.537635 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:51:47.537818 24427 external_mini_cluster.cc:1428] Setting key 67ef75b9f2bd42f37b72a8655e5417b5
I20260809 18:51:47.541414 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.536265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:50971 (local address 127.23.218.254:39039)
0809 18:51:47.536310 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.536312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.536394 (+ 82us) server_negotiation.cc:407] Connection header received
0809 18:51:47.536461 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.536462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.536469 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.536511 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:47.536801 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.537308 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.538256 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.538492 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.538551 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.539385 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:47.539386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:47.539387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:47.539392 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:47.540437 (+ 1045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.540737 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.540739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.540739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.540789 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.540949 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.540950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.540950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.541049 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:47.541114 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.541260 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.541328 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:51:47.541428 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.536218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:39039 (local address 127.23.218.193:50971)
0809 18:51:47.536341 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.536352 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.536409 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.536515 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.536516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.536688 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:47.536747 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.536749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.537404 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.537405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.538187 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.538190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.538551 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.538551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.538623 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.538951 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:47.538956 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:47.539311 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:47.540510 (+ 1199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.540511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.540512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.540668 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.540841 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.540841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.540842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.540900 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.541105 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:47.541106 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:47.541159 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.541231 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.541329 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:51:47.564316 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:39039
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:47.642900 27567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:47.643075 27567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:47.643107 27567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:47.644260 27567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:47.644300 27567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:47.644327 27567 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 18:51:47.644342 27567 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 18:51:47.644367 27567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:51:47.645795 27567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.218.254:39039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:47.646021 27567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:47.646210 27567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:47.650808 27573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:47.650863 27567 server_base.cc:1034] running on GCE node
W20260809 18:51:47.650820 27577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:47.650842 27574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:47.651194 27567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:47.651412 27567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:47.652546 27567 hybrid_clock.cc:648] HybridClock initialized: now 1786301507652531 us; error 27 us; skew 500 ppm
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:47.654760 27567 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:51:47.655133 27567 webserver.cc:460] Webserver started at http://127.23.218.194:45551/ using document root <none> and password file <none>
I20260809 18:51:47.655336 27567 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:47.655383 27567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:47.655498 27567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:47.656432 27567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "51fc1b4f06de43fc829eb37efb336782"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "afcfa4162bc77a065343b7e8fea0bf05"
server_key_iv: "6ac8922cc8989d20e6b3be145ccb8f57"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.656752 27567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "51fc1b4f06de43fc829eb37efb336782"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "afcfa4162bc77a065343b7e8fea0bf05"
server_key_iv: "6ac8922cc8989d20e6b3be145ccb8f57"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.657902 27567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:51:47.658548 27586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.658692 27567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:47.658782 27567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "51fc1b4f06de43fc829eb37efb336782"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "afcfa4162bc77a065343b7e8fea0bf05"
server_key_iv: "6ac8922cc8989d20e6b3be145ccb8f57"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.658838 27567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:47.667337 27567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:47.668848 27567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:47.668969 27567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:47.669396 27567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:47.669435 27567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.669461 27567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:47.669476 27567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.675540 27567 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:34589
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:47.675655 27784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:34589 every 8 connection(s)
I20260809 18:51:47.675871 27567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:51:47.678330 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.669767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:45567 (local address 127.23.218.254:39039)
0809 18:51:47.669831 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.669834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.670045 (+ 211us) server_negotiation.cc:407] Connection header received
0809 18:51:47.670514 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.670515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.670528 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.670573 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:47.671316 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.672042 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.673470 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.673765 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.673829 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.676091 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:47.676093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:47.676094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:47.676102 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:47.677150 (+ 1048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.677491 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.677492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.677492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.677546 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.677760 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.677761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.677762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.677868 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:47.677922 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.678169 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.678245 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 18:51:47.678514 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.669735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:39039 (local address 127.23.218.194:45567)
0809 18:51:47.669955 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.669979 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.670413 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.670649 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.670653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.670978 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:47.671228 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.671234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.672155 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.672157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.673385 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.673388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.673838 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.673840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.673971 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.674423 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:47.674434 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:47.676010 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:47.677220 (+ 1210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.677226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.677228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.677426 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.677599 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.677599 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.677599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.677704 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.677922 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:47.677928 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:47.678047 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.678131 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.678273 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":70,"threads_started":1}
I20260809 18:51:47.678627 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27567
I20260809 18:51:47.678699 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:51:47.678874 24427 external_mini_cluster.cc:1428] Setting key 85e58e3c01ed502c79699dc2d48a952f
I20260809 18:51:47.678993 27787 heartbeater.cc:347] Connected to a master server at 127.23.218.254:39039
I20260809 18:51:47.679072 27787 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:47.679239 27787 heartbeater.cc:511] Master 127.23.218.254:39039 requested a full tablet report, sending...
I20260809 18:51:47.679579 27209 ts_manager.cc:194] Registered new tserver with Master: 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589)
I20260809 18:51:47.680024 27209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:45567
I20260809 18:51:47.684805 27793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.679881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:39039 (local address 127.23.218.194:41779)
0809 18:51:47.680002 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.680011 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.680058 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.680174 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.680174 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.680319 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:47.680371 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.680372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.681095 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.681095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.681579 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.681580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.681971 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.681971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.682048 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.682402 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:47.682407 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:47.682673 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:47.683902 (+ 1229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.683903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.683903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.684060 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.684223 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.684223 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.684224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.684279 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.684561 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:47.684562 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:47.684599 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.684670 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.684741 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260809 18:51:47.684875 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.679921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:41779 (local address 127.23.218.254:39039)
0809 18:51:47.679958 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.679960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.680064 (+ 104us) server_negotiation.cc:407] Connection header received
0809 18:51:47.680102 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.680102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.680111 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.680169 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:47.680431 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.681012 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.681652 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.681896 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.681952 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.682751 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:47.682753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:47.682753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:47.682760 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:47.683833 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.684119 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.684120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.684120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.684169 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.684328 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.684329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.684329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.684486 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:47.684552 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.684717 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.684792 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:51:47.701726 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:39039
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:47.795459 27800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:47.795634 27800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:47.795655 27800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:47.796728 27800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:47.796766 27800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:47.796788 27800 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 18:51:47.796813 27800 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 18:51:47.796839 27800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:51:47.798249 27800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.218.254:39039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:47.798467 27800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:47.798653 27800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:47.803308 27816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:47.803320 27809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:47.803344 27811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:47.803730 27800 server_base.cc:1034] running on GCE node
I20260809 18:51:47.803936 27800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:47.804185 27800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:47.805418 27800 hybrid_clock.cc:648] HybridClock initialized: now 1786301507805394 us; error 39 us; skew 500 ppm
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:47.811328 27800 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:51:47.811713 27800 webserver.cc:460] Webserver started at http://127.23.218.195:38509/ using document root <none> and password file <none>
I20260809 18:51:47.811916 27800 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:47.811962 27800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:47.812083 27800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:47.813047 27800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "a53dbd27a4494ba1a0b16c36f50c76e7"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "223487aa290499e10efa4127bfcfca3d"
server_key_iv: "385c34dc09f8c1cc4e6af68e956c3252"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.813378 27800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "a53dbd27a4494ba1a0b16c36f50c76e7"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "223487aa290499e10efa4127bfcfca3d"
server_key_iv: "385c34dc09f8c1cc4e6af68e956c3252"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.814638 27800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:51:47.815330 27824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.815479 27800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:47.815568 27800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "a53dbd27a4494ba1a0b16c36f50c76e7"
format_stamp: "Formatted at 2026-08-09 18:51:47 on dist-test-slave-6zbq"
server_key: "223487aa290499e10efa4127bfcfca3d"
server_key_iv: "385c34dc09f8c1cc4e6af68e956c3252"
server_key_version: "encryptionkey@0"
I20260809 18:51:47.815656 27800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:47.831429 27800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:47.833312 27800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:47.833436 27800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:47.833938 27800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:47.833995 27800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.834022 27800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:47.834048 27800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:47.841028 27800 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:36093
I20260809 18:51:47.841398 27800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:51:47.841846 28008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:36093 every 8 connection(s)
I20260809 18:51:47.843917 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.834259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:43825 (local address 127.23.218.254:39039)
0809 18:51:47.834304 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.834306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.834574 (+ 268us) server_negotiation.cc:407] Connection header received
0809 18:51:47.835186 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.835188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.835209 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.835260 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:47.835995 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.836750 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.838168 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.838532 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.838607 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.841050 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:47.841051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:47.841052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:47.841060 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:47.842543 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.843024 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.843024 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.843025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.843070 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.843331 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.843332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.843332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.843426 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:47.843473 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.843799 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.843859 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 18:51:47.844151 27840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.834270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:39039 (local address 127.23.218.195:43825)
0809 18:51:47.834432 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.834477 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.835062 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.835336 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.835340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.835660 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:47.835912 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.835918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.836862 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.836863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.838089 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.838093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.838624 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.838625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.838774 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.839265 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:47.839275 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:47.840946 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:47.842657 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.842664 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.842666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.842958 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.843112 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.843113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.843113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.843255 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.843488 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:47.843494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:47.843661 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.843782 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.843937 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260809 18:51:47.844606 28010 heartbeater.cc:347] Connected to a master server at 127.23.218.254:39039
I20260809 18:51:47.844689 28010 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:47.844858 28010 heartbeater.cc:511] Master 127.23.218.254:39039 requested a full tablet report, sending...
I20260809 18:51:47.845184 27208 ts_manager.cc:194] Registered new tserver with Master: a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093)
I20260809 18:51:47.845563 27208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:43825
I20260809 18:51:47.848471 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27800
I20260809 18:51:47.848577 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:51:47.848760 24427 external_mini_cluster.cc:1428] Setting key 081ead80032eb3cb24d06b0d95e5e017
I20260809 18:51:47.849888 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:51:47.851106 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.845742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:53283 (local address 127.23.218.254:39039)
0809 18:51:47.845773 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.845775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.845867 (+ 92us) server_negotiation.cc:407] Connection header received
0809 18:51:47.845952 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.845953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.845962 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.845988 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:47.846381 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.846829 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.847435 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.847667 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.847722 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.848554 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:47.848556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:47.848556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:47.848563 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:47.849963 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.850333 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.850334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.850335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.850408 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.850601 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.850602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.850603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.850725 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:47.850764 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.850959 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.851029 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 18:51:47.851547 28024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.845631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:39039 (local address 127.23.218.195:53283)
0809 18:51:47.845811 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.845820 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.845890 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.846040 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.846040 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.846221 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:47.846282 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.846284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.846903 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.846904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.847373 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.847374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.847734 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.847735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.847806 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.848113 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:47.848116 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:47.848450 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:47.850073 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.850074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.850074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.850255 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.850478 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.850479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.850479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.850536 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.850808 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:47.850809 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:47.850848 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.851365 (+ 517us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.851463 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:47.879148 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.871976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.23.218.254:39039)
0809 18:51:47.872041 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.872043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.872052 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:51:47.872082 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.872083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.872095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.872171 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:47.872693 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.873455 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.874230 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.874567 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.874638 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.876640 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:47.876641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:47.876642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:47.876649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:47.878031 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.878395 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.878397 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.878397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.878478 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.878678 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.878679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.878679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.878807 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:47.878897 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.878968 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.879046 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 18:51:47.880772 27208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:51:47.883030 27208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:51:47.890313 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.885382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:36093 (local address 127.0.0.1:51450)
0809 18:51:47.885885 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.885893 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.885931 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.886444 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.886445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.886447 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:47.886509 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.886511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.887770 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.887771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.889131 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.889134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.890029 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.890029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.890115 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.890130 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.890176 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.890244 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":72,"thread_start_us":72,"threads_started":1}
I20260809 18:51:47.890367 28045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.885611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55262 (local address 127.23.218.194:34589)
0809 18:51:47.885849 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.885854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.886210 (+ 356us) server_negotiation.cc:407] Connection header received
0809 18:51:47.886248 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.886250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.886263 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.886328 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:47.886623 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.887728 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.889226 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.889935 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.890012 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.890072 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.890236 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.890275 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260809 18:51:47.890501 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.885386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41519 (local address 127.0.0.1:37126)
0809 18:51:47.886018 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.886026 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.886061 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.886374 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.886378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.886381 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:47.886461 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.886464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.887736 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.887737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.889205 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.889209 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.890185 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.890187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.890304 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.890309 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.890361 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.890430 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":96,"threads_started":1}
I20260809 18:51:47.890625 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.885382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34589 (local address 127.0.0.1:55262)
0809 18:51:47.885779 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.885826 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.885919 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.886513 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.886514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.886516 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:47.886551 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.886553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.887803 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.887804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.889131 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.889134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.890102 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.890103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.890152 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.890155 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.890196 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.890488 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":280,"thread_start_us":62,"threads_started":1}
I20260809 18:51:47.890877 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.885611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.23.218.195:36093)
0809 18:51:47.886064 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.886068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.886078 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:51:47.886115 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.886118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.886133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.886446 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:47.886589 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.887666 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.889230 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.889928 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.890654 (+ 726us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.890719 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.890746 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.890783 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":84,"threads_started":1}
I20260809 18:51:47.890944 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.885644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37126 (local address 127.23.218.193:41519)
0809 18:51:47.886172 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.886190 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.886199 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:51:47.886235 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.886237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.886251 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.886325 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:47.886560 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.887629 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.889314 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.890029 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.890722 (+ 693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.890795 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.890820 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.890859 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":78,"threads_started":1}
I20260809 18:51:47.891304 27673 tablet_service.cc:1467] Processing CreateTablet for tablet ce2c75bfd9644fec8447066e8b39764b (DEFAULT_TABLE table=test-table [id=54fd16aed8ce45deb3816bcd1cd94d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:47.891487 27440 tablet_service.cc:1467] Processing CreateTablet for tablet ce2c75bfd9644fec8447066e8b39764b (DEFAULT_TABLE table=test-table [id=54fd16aed8ce45deb3816bcd1cd94d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:47.891573 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce2c75bfd9644fec8447066e8b39764b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:47.891724 27440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce2c75bfd9644fec8447066e8b39764b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:47.893594 27906 tablet_service.cc:1467] Processing CreateTablet for tablet ce2c75bfd9644fec8447066e8b39764b (DEFAULT_TABLE table=test-table [id=54fd16aed8ce45deb3816bcd1cd94d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:47.893927 27906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce2c75bfd9644fec8447066e8b39764b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:47.894273 28047 tablet_bootstrap.cc:492] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: Bootstrap starting.
I20260809 18:51:47.894794 28048 tablet_bootstrap.cc:492] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: Bootstrap starting.
I20260809 18:51:47.894887 28047 tablet_bootstrap.cc:654] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:47.895208 28047 log.cc:824] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:47.895392 28048 tablet_bootstrap.cc:654] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:47.895644 28048 log.cc:824] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:47.896375 28048 tablet_bootstrap.cc:492] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: No bootstrap required, opened a new log
I20260809 18:51:47.896433 28048 ts_tablet_manager.cc:1397] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:51:47.896667 28052 tablet_bootstrap.cc:492] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: Bootstrap starting.
I20260809 18:51:47.897224 28047 tablet_bootstrap.cc:492] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: No bootstrap required, opened a new log
I20260809 18:51:47.897300 28047 ts_tablet_manager.cc:1397] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 18:51:47.897469 28052 tablet_bootstrap.cc:654] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:47.897733 28052 log.cc:824] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:47.898310 28052 tablet_bootstrap.cc:492] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: No bootstrap required, opened a new log
I20260809 18:51:47.898403 28052 ts_tablet_manager.cc:1397] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:51:47.898366 28048 raft_consensus.cc:348] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } }
I20260809 18:51:47.898537 28048 raft_consensus.cc:374] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:47.898579 28048 raft_consensus.cc:729] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e728dc64844133b2b732a7b36a991a, State: Initialized, Role: FOLLOWER
I20260809 18:51:47.898690 28048 consensus_queue.cc:260] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } }
I20260809 18:51:47.898939 28048 ts_tablet_manager.cc:1428] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 18:51:47.899066 27553 heartbeater.cc:503] Master 127.23.218.254:39039 was elected leader, sending a full tablet report...
I20260809 18:51:47.899179 28047 raft_consensus.cc:348] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } }
I20260809 18:51:47.899284 28047 raft_consensus.cc:374] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:47.899303 28047 raft_consensus.cc:729] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51fc1b4f06de43fc829eb37efb336782, State: Initialized, Role: FOLLOWER
I20260809 18:51:47.899382 28047 consensus_queue.cc:260] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } }
I20260809 18:51:47.899586 28047 ts_tablet_manager.cc:1428] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260809 18:51:47.899695 27787 heartbeater.cc:503] Master 127.23.218.254:39039 was elected leader, sending a full tablet report...
I20260809 18:51:47.900239 28052 raft_consensus.cc:348] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } }
I20260809 18:51:47.900364 28052 raft_consensus.cc:374] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:47.900391 28052 raft_consensus.cc:729] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53dbd27a4494ba1a0b16c36f50c76e7, State: Initialized, Role: FOLLOWER
I20260809 18:51:47.900493 28052 consensus_queue.cc:260] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } }
I20260809 18:51:47.900748 28052 ts_tablet_manager.cc:1428] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 18:51:47.900784 28010 heartbeater.cc:503] Master 127.23.218.254:39039 was elected leader, sending a full tablet report...
W20260809 18:51:47.926362 27788 tablet.cc:2369] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:51:47.947082 28058 raft_consensus.cc:482] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:47.947181 28058 raft_consensus.cc:504] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } }
I20260809 18:51:47.947492 28058 leader_election.cc:288] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093), 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589)
I20260809 18:51:47.951128 28045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.947732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:52951 (local address 127.23.218.194:34589)
0809 18:51:47.947767 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.947768 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.947900 (+ 132us) server_negotiation.cc:407] Connection header received
0809 18:51:47.947925 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.947925 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.947932 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.947972 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:47.948113 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.948806 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.950192 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.950784 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.950861 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.950870 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.951024 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.951061 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:51:47.951134 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.947701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:35641 (local address 127.23.218.195:36093)
0809 18:51:47.947755 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.947757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.947788 (+ 31us) server_negotiation.cc:407] Connection header received
0809 18:51:47.947812 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.947812 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.947820 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.947853 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:47.948009 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.948763 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.950193 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.950784 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.950861 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.950870 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.951024 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.951062 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 18:51:47.951161 27370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.947649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:36093 (local address 127.23.218.193:35641)
0809 18:51:47.947700 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.947710 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.947751 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.947876 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.947876 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.947878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:47.947946 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.947948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.948835 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.948836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.950057 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.950061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.950868 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.950869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.950963 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.950965 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.951012 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.951051 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":49}
I20260809 18:51:47.951179 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.947659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34589 (local address 127.23.218.193:52951)
0809 18:51:47.947829 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.947837 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.947888 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.947992 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.947993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.947994 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:47.948048 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.948049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.948868 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.948868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.950057 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.950061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.950868 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.950869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.950947 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.950952 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.951012 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.951051 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":6,"thread_start_us":87,"threads_started":1}
I20260809 18:51:47.951502 27936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2c75bfd9644fec8447066e8b39764b" candidate_uuid: "d8e728dc64844133b2b732a7b36a991a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" is_pre_election: true
I20260809 18:51:47.951508 27702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2c75bfd9644fec8447066e8b39764b" candidate_uuid: "d8e728dc64844133b2b732a7b36a991a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fc1b4f06de43fc829eb37efb336782" is_pre_election: true
I20260809 18:51:47.951651 27936 raft_consensus.cc:2454] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8e728dc64844133b2b732a7b36a991a in term 0.
I20260809 18:51:47.951651 27702 raft_consensus.cc:2454] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8e728dc64844133b2b732a7b36a991a in term 0.
I20260809 18:51:47.951881 27360 leader_election.cc:302] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fc1b4f06de43fc829eb37efb336782, d8e728dc64844133b2b732a7b36a991a; no voters:
I20260809 18:51:47.952052 28058 raft_consensus.cc:2781] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:51:47.952111 28058 raft_consensus.cc:482] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:51:47.952127 28058 raft_consensus.cc:3037] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:47.952778 28058 raft_consensus.cc:504] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } }
I20260809 18:51:47.952896 28058 leader_election.cc:288] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [CANDIDATE]: Term 1 election: Requested vote from peers a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093), 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589)
I20260809 18:51:47.953042 27702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2c75bfd9644fec8447066e8b39764b" candidate_uuid: "d8e728dc64844133b2b732a7b36a991a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fc1b4f06de43fc829eb37efb336782"
I20260809 18:51:47.953085 27936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce2c75bfd9644fec8447066e8b39764b" candidate_uuid: "d8e728dc64844133b2b732a7b36a991a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7"
I20260809 18:51:47.953112 27702 raft_consensus.cc:3037] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:47.953147 27936 raft_consensus.cc:3037] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:47.953790 27936 raft_consensus.cc:2454] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8e728dc64844133b2b732a7b36a991a in term 1.
I20260809 18:51:47.954071 27702 raft_consensus.cc:2454] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8e728dc64844133b2b732a7b36a991a in term 1.
I20260809 18:51:47.954268 27362 leader_election.cc:302] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53dbd27a4494ba1a0b16c36f50c76e7, d8e728dc64844133b2b732a7b36a991a; no voters:
I20260809 18:51:47.954401 28058 raft_consensus.cc:2781] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:47.954552 28058 raft_consensus.cc:686] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 1 LEADER]: Becoming Leader. State: Replica: d8e728dc64844133b2b732a7b36a991a, State: Running, Role: LEADER
I20260809 18:51:47.954648 28058 consensus_queue.cc:237] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } }
I20260809 18:51:47.955382 27208 catalog_manager.cc:5495] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8e728dc64844133b2b732a7b36a991a (127.23.218.193). New cstate: current_term: 1 leader_uuid: "d8e728dc64844133b2b732a7b36a991a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } health_report { overall_health: HEALTHY } } }
Aug 09 18:51:47 dist-test-slave-6zbq krb5kdc[27132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301507, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:47.992894 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.986897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:39039 (local address 127.0.0.1:44264)
0809 18:51:47.987051 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:47.987060 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:47.987117 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:47.987222 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:47.987222 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:47.987493 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:47.987534 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.987536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.988106 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.988107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.988608 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:47.988609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.988925 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.988926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:47.988961 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.989433 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:47.989438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:47.990779 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:47.992080 (+ 1301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.992085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.992087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.992268 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.992422 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:47.992423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:47.992423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:47.992471 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:47.992635 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:47.992638 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:47.992677 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 18:51:47.992721 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 18:51:47.992808 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260809 18:51:47.992936 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:47.986943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44264 (local address 127.23.218.254:39039)
0809 18:51:47.986996 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:47.986998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:47.987112 (+ 114us) server_negotiation.cc:407] Connection header received
0809 18:51:47.987155 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:47.987156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:47.987165 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:47.987212 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:47.987594 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.988053 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.988685 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:47.988866 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:47.988925 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:47.990850 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:47.990852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:47.990852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:47.990858 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:47.992019 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.992326 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.992327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.992327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.992375 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:47.992509 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:47.992509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:47.992509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:47.992595 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:47.992639 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:47.992738 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 18:51:47.992850 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":28}
I20260809 18:51:47.994748 27208 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:44264:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 18:51:47.994920 27208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:51:47.996244 27906 tablet_service.cc:1467] Processing CreateTablet for tablet cd260f272d5744aea6163ec7f1ec0b80 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1ced5420c63541fba5986b6d51d08d0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 18:51:47.996236 27440 tablet_service.cc:1467] Processing CreateTablet for tablet cd260f272d5744aea6163ec7f1ec0b80 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1ced5420c63541fba5986b6d51d08d0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 18:51:47.996310 27673 tablet_service.cc:1467] Processing CreateTablet for tablet cd260f272d5744aea6163ec7f1ec0b80 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1ced5420c63541fba5986b6d51d08d0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 18:51:47.996402 27440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd260f272d5744aea6163ec7f1ec0b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:47.996402 27906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd260f272d5744aea6163ec7f1ec0b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:47.996454 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd260f272d5744aea6163ec7f1ec0b80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:47.997466 28047 tablet_bootstrap.cc:492] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: Bootstrap starting.
I20260809 18:51:47.997788 28052 tablet_bootstrap.cc:492] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: Bootstrap starting.
I20260809 18:51:47.997870 28047 tablet_bootstrap.cc:654] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:47.998315 28052 tablet_bootstrap.cc:654] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:47.998423 28048 tablet_bootstrap.cc:492] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: Bootstrap starting.
I20260809 18:51:47.998602 28047 tablet_bootstrap.cc:492] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: No bootstrap required, opened a new log
I20260809 18:51:47.998661 28047 ts_tablet_manager.cc:1397] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 18:51:47.998806 28048 tablet_bootstrap.cc:654] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:47.998808 28047 raft_consensus.cc:348] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:47.998857 28047 raft_consensus.cc:374] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:47.998881 28047 raft_consensus.cc:729] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51fc1b4f06de43fc829eb37efb336782, State: Initialized, Role: FOLLOWER
I20260809 18:51:47.998924 28047 consensus_queue.cc:260] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:47.999017 28047 ts_tablet_manager.cc:1428] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.999094 28060 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:47.999171 28060 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:47.999375 28052 tablet_bootstrap.cc:492] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: No bootstrap required, opened a new log
I20260809 18:51:47.999424 28052 ts_tablet_manager.cc:1397] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 18:51:47.999568 28052 raft_consensus.cc:348] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:47.999636 28052 raft_consensus.cc:374] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:47.999655 28052 raft_consensus.cc:729] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53dbd27a4494ba1a0b16c36f50c76e7, State: Initialized, Role: FOLLOWER
I20260809 18:51:47.999675 28048 tablet_bootstrap.cc:492] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: No bootstrap required, opened a new log
I20260809 18:51:47.999703 28052 consensus_queue.cc:260] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:47.999722 28048 ts_tablet_manager.cc:1397] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:47.999802 28052 ts_tablet_manager.cc:1428] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:47.999861 28048 raft_consensus.cc:348] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:47.999882 28063 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:47.999961 28048 raft_consensus.cc:374] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:47.999963 28063 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:47.999984 28048 raft_consensus.cc:729] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e728dc64844133b2b732a7b36a991a, State: Initialized, Role: FOLLOWER
I20260809 18:51:48.000026 28048 consensus_queue.cc:260] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.000116 28048 ts_tablet_manager.cc:1428] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:48.000299 28058 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:48.000375 28058 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 18:51:48.032929 27555 tablet.cc:2369] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:51:48.070391 28058 raft_consensus.cc:482] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:48.070451 28058 raft_consensus.cc:504] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.070564 28058 leader_election.cc:288] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589), a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093)
I20260809 18:51:48.070732 27702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd260f272d5744aea6163ec7f1ec0b80" candidate_uuid: "d8e728dc64844133b2b732a7b36a991a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fc1b4f06de43fc829eb37efb336782" is_pre_election: true
I20260809 18:51:48.070760 27936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd260f272d5744aea6163ec7f1ec0b80" candidate_uuid: "d8e728dc64844133b2b732a7b36a991a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" is_pre_election: true
I20260809 18:51:48.070839 27702 raft_consensus.cc:2454] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8e728dc64844133b2b732a7b36a991a in term 0.
I20260809 18:51:48.070850 27936 raft_consensus.cc:2454] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8e728dc64844133b2b732a7b36a991a in term 0.
I20260809 18:51:48.071003 27362 leader_election.cc:302] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53dbd27a4494ba1a0b16c36f50c76e7, d8e728dc64844133b2b732a7b36a991a; no voters:
I20260809 18:51:48.071094 28058 raft_consensus.cc:2781] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:51:48.071133 28058 raft_consensus.cc:482] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:51:48.071152 28058 raft_consensus.cc:3037] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:48.071712 28058 raft_consensus.cc:504] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.071813 28058 leader_election.cc:288] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [CANDIDATE]: Term 1 election: Requested vote from peers 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589), a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093)
I20260809 18:51:48.071964 27702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd260f272d5744aea6163ec7f1ec0b80" candidate_uuid: "d8e728dc64844133b2b732a7b36a991a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fc1b4f06de43fc829eb37efb336782"
I20260809 18:51:48.072031 27702 raft_consensus.cc:3037] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:48.072010 27936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd260f272d5744aea6163ec7f1ec0b80" candidate_uuid: "d8e728dc64844133b2b732a7b36a991a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7"
I20260809 18:51:48.072088 27936 raft_consensus.cc:3037] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:48.072584 27702 raft_consensus.cc:2454] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8e728dc64844133b2b732a7b36a991a in term 1.
I20260809 18:51:48.072701 27936 raft_consensus.cc:2454] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8e728dc64844133b2b732a7b36a991a in term 1.
I20260809 18:51:48.072744 27360 leader_election.cc:302] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fc1b4f06de43fc829eb37efb336782, d8e728dc64844133b2b732a7b36a991a; no voters:
I20260809 18:51:48.072844 28058 raft_consensus.cc:2781] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:48.072906 28058 raft_consensus.cc:686] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [term 1 LEADER]: Becoming Leader. State: Replica: d8e728dc64844133b2b732a7b36a991a, State: Running, Role: LEADER
I20260809 18:51:48.072962 28058 consensus_queue.cc:237] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.073217 28065 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: TxnStatusTablet state changed. Reason: New leader d8e728dc64844133b2b732a7b36a991a. Latest consensus state: current_term: 1 leader_uuid: "d8e728dc64844133b2b732a7b36a991a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:48.073284 28065 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: This TxnStatusTablet replica's current role is: LEADER
I20260809 18:51:48.073413 28089 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 18:51:48.073526 28089 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 18:51:48.073478 27208 catalog_manager.cc:5495] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8e728dc64844133b2b732a7b36a991a (127.23.218.193). New cstate: current_term: 1 leader_uuid: "d8e728dc64844133b2b732a7b36a991a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } health_report { overall_health: UNKNOWN } } }
I20260809 18:51:48.092028 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.089347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37136 (local address 127.23.218.193:41519)
0809 18:51:48.089399 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:48.089401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:48.089411 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:51:48.089474 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:48.089474 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:48.089494 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:48.089560 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:48.089658 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.090417 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.091166 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.091468 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.091533 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.091671 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.091789 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.091863 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:48.091916 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 18:51:48.091937 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 18:51:48.092043 28011 tablet.cc:2369] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:51:48.092037 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.089256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41519 (local address 127.0.0.1:37136)
0809 18:51:48.089346 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:48.089357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:48.089407 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:48.089565 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:48.089565 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:48.089567 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:51:48.089615 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.089617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.090512 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.090513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.091090 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.091093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.091529 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.091529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.091573 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.091590 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.091859 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.091869 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 18:51:48.091911 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 18:51:48.091965 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 18:51:48.093793 27936 raft_consensus.cc:1261] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Refusing update from remote peer d8e728dc64844133b2b732a7b36a991a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:48.093832 27702 raft_consensus.cc:1261] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Refusing update from remote peer d8e728dc64844133b2b732a7b36a991a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:48.094139 28058 consensus_queue.cc:1035] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:48.094264 28065 consensus_queue.cc:1035] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:48.095526 28060 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: TxnStatusTablet state changed. Reason: New leader d8e728dc64844133b2b732a7b36a991a. Latest consensus state: current_term: 1 leader_uuid: "d8e728dc64844133b2b732a7b36a991a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:48.095546 28063 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: TxnStatusTablet state changed. Reason: New leader d8e728dc64844133b2b732a7b36a991a. Latest consensus state: current_term: 1 leader_uuid: "d8e728dc64844133b2b732a7b36a991a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:48.095613 28060 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:48.095645 28063 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:48.096540 28065 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8e728dc64844133b2b732a7b36a991a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:48.096568 28060 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8e728dc64844133b2b732a7b36a991a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:48.096643 28065 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: This TxnStatusTablet replica's current role is: LEADER
I20260809 18:51:48.096657 28060 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P 51fc1b4f06de43fc829eb37efb336782: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:48.096817 28094 mvcc.cc:205] Tried to move back new op lower bound from 7316690977149960192 to 7316690977067249664. Current Snapshot: MvccSnapshot[applied={T|T < 7316690977149960192}]
I20260809 18:51:48.097244 28096 mvcc.cc:205] Tried to move back new op lower bound from 7316690977149960192 to 7316690977067249664. Current Snapshot: MvccSnapshot[applied={T|T < 7316690977149960192}]
I20260809 18:51:48.097285 28065 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8e728dc64844133b2b732a7b36a991a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:48.097352 28065 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P d8e728dc64844133b2b732a7b36a991a: This TxnStatusTablet replica's current role is: LEADER
I20260809 18:51:48.097587 28063 tablet_replica.cc:439] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8e728dc64844133b2b732a7b36a991a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } } }
I20260809 18:51:48.097656 28063 tablet_replica.cc:442] T cd260f272d5744aea6163ec7f1ec0b80 P a53dbd27a4494ba1a0b16c36f50c76e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:51:48.098330 28095 mvcc.cc:205] Tried to move back new op lower bound from 7316690977149960192 to 7316690977067249664. Current Snapshot: MvccSnapshot[applied={T|T < 7316690977149960192 or (T in {7316690977149960192})}]
I20260809 18:51:48.101624 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.099313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.23.218.193:41519)
0809 18:51:48.099347 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:48.099349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:48.099356 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:51:48.099419 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:48.099420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:48.099440 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:48.099483 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:48.099591 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.100001 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.101063 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.101266 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.101309 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.101452 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.101476 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.101515 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:48.101565 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 18:51:48.101581 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:51:48.105938 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.103815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:59109 (local address 127.23.218.193:41519)
0809 18:51:48.103844 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:48.103846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:48.103963 (+ 117us) server_negotiation.cc:407] Connection header received
0809 18:51:48.104016 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:48.104016 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:48.104024 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:48.104067 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:48.104181 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.104625 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.105345 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.105513 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.105562 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.105731 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.105759 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.105802 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:48.105856 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 18:51:48.105869 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 18:51:48.105993 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.103777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41519 (local address 127.23.218.193:59109)
0809 18:51:48.103905 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:48.103914 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:48.103966 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:48.104077 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:48.104078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:48.104079 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:51:48.104131 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.104132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.104692 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.104693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.105275 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.105276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.105567 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.105568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.105643 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.105656 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.105808 (+ 152us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.105809 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 18:51:48.105845 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 18:51:48.105881 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
I20260809 18:51:48.108220 27936 raft_consensus.cc:1261] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Refusing update from remote peer d8e728dc64844133b2b732a7b36a991a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:48.108230 27702 raft_consensus.cc:1261] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Refusing update from remote peer d8e728dc64844133b2b732a7b36a991a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:48.108405 28101 consensus_queue.cc:1035] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:48.108489 28065 consensus_queue.cc:1035] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [LEADER]: Connected to new peer: Peer: permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:48.120173 27208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:48.120270 27208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:48.120949 27208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76: Sending DeleteTablet for 3 replicas of tablet ce2c75bfd9644fec8447066e8b39764b
I20260809 18:51:48.121232 27906 tablet_service.cc:1514] Processing DeleteTablet for tablet ce2c75bfd9644fec8447066e8b39764b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:48 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:51450
I20260809 18:51:48.121254 27673 tablet_service.cc:1514] Processing DeleteTablet for tablet ce2c75bfd9644fec8447066e8b39764b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:48 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:55262
I20260809 18:51:48.121481 28122 tablet_replica.cc:330] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: stopping tablet replica
I20260809 18:51:48.121475 27438 tablet_service.cc:1514] Processing DeleteTablet for tablet ce2c75bfd9644fec8447066e8b39764b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:48 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:37126
I20260809 18:51:48.121477 27208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:51:48.121608 27208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:51:48.121688 28119 tablet_replica.cc:330] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: stopping tablet replica
I20260809 18:51:48.121848 28122 raft_consensus.cc:2229] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:48.121923 28122 raft_consensus.cc:2258] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:48.122713 27438 tablet_service.cc:1467] Processing CreateTablet for tablet a9018f1b10724ead996b09b9b4e8ee84 (DEFAULT_TABLE table=test-table [id=eef15c48f98749dab6b45a45088e41c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:48.122714 27906 tablet_service.cc:1467] Processing CreateTablet for tablet a9018f1b10724ead996b09b9b4e8ee84 (DEFAULT_TABLE table=test-table [id=eef15c48f98749dab6b45a45088e41c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:48.122833 27906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9018f1b10724ead996b09b9b4e8ee84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:48.122817 27673 tablet_service.cc:1467] Processing CreateTablet for tablet a9018f1b10724ead996b09b9b4e8ee84 (DEFAULT_TABLE table=test-table [id=eef15c48f98749dab6b45a45088e41c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:48.122836 27438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9018f1b10724ead996b09b9b4e8ee84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:48.122905 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9018f1b10724ead996b09b9b4e8ee84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:48.124074 28052 tablet_bootstrap.cc:492] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: Bootstrap starting.
I20260809 18:51:48.124125 28048 tablet_bootstrap.cc:492] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: Bootstrap starting.
I20260809 18:51:48.124125 28047 tablet_bootstrap.cc:492] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: Bootstrap starting.
I20260809 18:51:48.124434 28052 tablet_bootstrap.cc:654] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:48.124495 28047 tablet_bootstrap.cc:654] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:48.124495 28048 tablet_bootstrap.cc:654] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:48.125018 28122 ts_tablet_manager.cc:1905] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:48.125162 28119 raft_consensus.cc:2229] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:48.125252 28119 raft_consensus.cc:2258] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:48.125648 28119 ts_tablet_manager.cc:1905] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:48.126588 28122 ts_tablet_manager.cc:1918] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:51:48.126662 28122 log.cc:1196] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/ce2c75bfd9644fec8447066e8b39764b
I20260809 18:51:48.126891 28122 ts_tablet_manager.cc:1939] T ce2c75bfd9644fec8447066e8b39764b P 51fc1b4f06de43fc829eb37efb336782: Deleting consensus metadata
I20260809 18:51:48.127301 27186 catalog_manager.cc:4841] TS 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589): tablet ce2c75bfd9644fec8447066e8b39764b (table test-table [id=54fd16aed8ce45deb3816bcd1cd94d32]) successfully deleted
I20260809 18:51:48.127432 28119 ts_tablet_manager.cc:1918] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:51:48.127489 28119 log.cc:1196] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/ce2c75bfd9644fec8447066e8b39764b
I20260809 18:51:48.127693 28119 ts_tablet_manager.cc:1939] T ce2c75bfd9644fec8447066e8b39764b P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting consensus metadata
I20260809 18:51:48.128088 27188 catalog_manager.cc:4841] TS a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093): tablet ce2c75bfd9644fec8447066e8b39764b (table test-table [id=54fd16aed8ce45deb3816bcd1cd94d32]) successfully deleted
I20260809 18:51:48.130429 28124 tablet_replica.cc:330] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: stopping tablet replica
I20260809 18:51:48.130532 28124 raft_consensus.cc:2229] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:51:48.130627 28124 raft_consensus.cc:2258] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:48.130957 28047 tablet_bootstrap.cc:492] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: No bootstrap required, opened a new log
I20260809 18:51:48.131006 28047 ts_tablet_manager.cc:1397] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20260809 18:51:48.131014 28124 ts_tablet_manager.cc:1905] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:48.131254 28052 tablet_bootstrap.cc:492] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: No bootstrap required, opened a new log
I20260809 18:51:48.131254 28047 raft_consensus.cc:348] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.131297 28052 ts_tablet_manager.cc:1397] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260809 18:51:48.131320 28047 raft_consensus.cc:374] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:48.131340 28047 raft_consensus.cc:729] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51fc1b4f06de43fc829eb37efb336782, State: Initialized, Role: FOLLOWER
I20260809 18:51:48.131387 28047 consensus_queue.cc:260] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.131430 28052 raft_consensus.cc:348] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.131475 28047 ts_tablet_manager.cc:1428] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:48.131481 28052 raft_consensus.cc:374] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:48.131500 28052 raft_consensus.cc:729] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53dbd27a4494ba1a0b16c36f50c76e7, State: Initialized, Role: FOLLOWER
I20260809 18:51:48.131541 28052 consensus_queue.cc:260] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.131614 28052 ts_tablet_manager.cc:1428] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:48.131673 28048 tablet_bootstrap.cc:492] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: No bootstrap required, opened a new log
I20260809 18:51:48.131712 28048 ts_tablet_manager.cc:1397] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260809 18:51:48.131793 28048 raft_consensus.cc:348] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.131865 28048 raft_consensus.cc:374] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:48.131884 28048 raft_consensus.cc:729] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e728dc64844133b2b732a7b36a991a, State: Initialized, Role: FOLLOWER
I20260809 18:51:48.131919 28048 consensus_queue.cc:260] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.131992 28048 ts_tablet_manager.cc:1428] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:48.132503 28124 ts_tablet_manager.cc:1918] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:51:48.132570 28124 log.cc:1196] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/ce2c75bfd9644fec8447066e8b39764b
I20260809 18:51:48.132809 28124 ts_tablet_manager.cc:1939] T ce2c75bfd9644fec8447066e8b39764b P d8e728dc64844133b2b732a7b36a991a: Deleting consensus metadata
I20260809 18:51:48.133134 27187 catalog_manager.cc:4841] TS d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519): tablet ce2c75bfd9644fec8447066e8b39764b (table test-table [id=54fd16aed8ce45deb3816bcd1cd94d32]) successfully deleted
I20260809 18:51:48.230551 28060 raft_consensus.cc:482] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:48.230665 28060 raft_consensus.cc:504] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.231026 28060 leader_election.cc:288] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519), a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093)
I20260809 18:51:48.234397 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.231316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:38207 (local address 127.23.218.195:36093)
0809 18:51:48.231405 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:48.231406 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:48.231444 (+ 38us) server_negotiation.cc:407] Connection header received
0809 18:51:48.231559 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:48.231560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:48.231570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:48.231615 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:48.231802 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.232540 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.233664 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.234019 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.234079 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.234089 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:48.234251 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 18:51:48.234304 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 18:51:48.234421 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.231188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41519 (local address 127.23.218.194:53005)
0809 18:51:48.231347 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:48.231359 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:48.231442 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:48.231581 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:48.231582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:48.231584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:48.231657 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.231659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.232281 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.232282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.233339 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.233341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.234099 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.234100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.234171 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.234175 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:51:48.234230 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 18:51:48.234273 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260809 18:51:48.234639 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.231292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:53005 (local address 127.23.218.193:41519)
0809 18:51:48.231370 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:48.231373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:48.231412 (+ 39us) server_negotiation.cc:407] Connection header received
0809 18:51:48.231505 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:48.231506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:48.231515 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:48.231560 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:48.231728 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.232205 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.233432 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.234002 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.234078 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.234089 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:48.234536 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 18:51:48.234577 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 18:51:48.234421 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.231190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:36093 (local address 127.23.218.194:38207)
0809 18:51:48.231369 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:48.231380 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:48.231473 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:48.231634 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:48.231635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:48.231638 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:48.231708 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.231711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.232650 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.232651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.233587 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.233589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.234117 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.234118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.234178 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.234180 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:51:48.234230 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 18:51:48.234272 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":60,"threads_started":1}
I20260809 18:51:48.234886 27471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9018f1b10724ead996b09b9b4e8ee84" candidate_uuid: "51fc1b4f06de43fc829eb37efb336782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e728dc64844133b2b732a7b36a991a" is_pre_election: true
I20260809 18:51:48.235018 27471 raft_consensus.cc:2454] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51fc1b4f06de43fc829eb37efb336782 in term 0.
I20260809 18:51:48.235064 27936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9018f1b10724ead996b09b9b4e8ee84" candidate_uuid: "51fc1b4f06de43fc829eb37efb336782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" is_pre_election: true
I20260809 18:51:48.235172 27936 raft_consensus.cc:2454] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51fc1b4f06de43fc829eb37efb336782 in term 0.
I20260809 18:51:48.235212 27589 leader_election.cc:302] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fc1b4f06de43fc829eb37efb336782, d8e728dc64844133b2b732a7b36a991a; no voters:
I20260809 18:51:48.235426 28060 raft_consensus.cc:2781] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:51:48.235477 28060 raft_consensus.cc:482] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:51:48.235498 28060 raft_consensus.cc:3037] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:48.236407 28060 raft_consensus.cc:504] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.236542 28060 leader_election.cc:288] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [CANDIDATE]: Term 1 election: Requested vote from peers d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519), a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093)
I20260809 18:51:48.236716 27471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9018f1b10724ead996b09b9b4e8ee84" candidate_uuid: "51fc1b4f06de43fc829eb37efb336782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e728dc64844133b2b732a7b36a991a"
I20260809 18:51:48.236716 27936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9018f1b10724ead996b09b9b4e8ee84" candidate_uuid: "51fc1b4f06de43fc829eb37efb336782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7"
I20260809 18:51:48.236788 27471 raft_consensus.cc:3037] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:48.236792 27936 raft_consensus.cc:3037] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:48.237407 27471 raft_consensus.cc:2454] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51fc1b4f06de43fc829eb37efb336782 in term 1.
I20260809 18:51:48.237488 27936 raft_consensus.cc:2454] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51fc1b4f06de43fc829eb37efb336782 in term 1.
I20260809 18:51:48.237562 27589 leader_election.cc:302] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fc1b4f06de43fc829eb37efb336782, d8e728dc64844133b2b732a7b36a991a; no voters:
I20260809 18:51:48.237648 28060 raft_consensus.cc:2781] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:48.237771 28060 raft_consensus.cc:686] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 1 LEADER]: Becoming Leader. State: Replica: 51fc1b4f06de43fc829eb37efb336782, State: Running, Role: LEADER
I20260809 18:51:48.237840 28060 consensus_queue.cc:237] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:48.238507 27208 catalog_manager.cc:5495] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194). New cstate: current_term: 1 leader_uuid: "51fc1b4f06de43fc829eb37efb336782" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } health_report { overall_health: UNKNOWN } } }
I20260809 18:51:48.274113 28045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.271591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55276 (local address 127.23.218.194:34589)
0809 18:51:48.271643 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:48.271645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:48.271652 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:51:48.271681 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:48.271681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:48.271698 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:48.271755 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:48.271917 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.272388 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.273396 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.273576 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.273632 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.273814 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.273915 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.273964 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:48.274037 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 18:51:48.274053 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 18:51:48.278843 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.276639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:35073 (local address 127.23.218.193:41519)
0809 18:51:48.276675 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:48.276677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:48.276798 (+ 121us) server_negotiation.cc:407] Connection header received
0809 18:51:48.276851 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:48.276852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:48.276861 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:48.276900 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:48.277024 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.277454 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.278182 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.278416 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.278463 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.278636 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.278669 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.278713 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:48.278764 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 18:51:48.278781 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:51:48.278904 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.276578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41519 (local address 127.23.218.194:35073)
0809 18:51:48.276734 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:48.276744 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:48.276797 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:48.276911 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:48.276911 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:48.276913 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:51:48.276974 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.276975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.277524 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.277525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.278109 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.278111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.278482 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.278483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.278550 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.278563 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.278722 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.278724 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:51:48.278779 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 18:51:48.278820 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260809 18:51:48.281324 27936 raft_consensus.cc:1261] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Refusing update from remote peer 51fc1b4f06de43fc829eb37efb336782: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:48.281415 27471 raft_consensus.cc:1261] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Refusing update from remote peer 51fc1b4f06de43fc829eb37efb336782: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:48.281639 28134 consensus_queue.cc:1035] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:48.281785 28060 consensus_queue.cc:1035] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:48.289148 28045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.286609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:56833 (local address 127.23.218.194:34589)
0809 18:51:48.286664 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:48.286666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:48.286673 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:51:48.286690 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:48.286691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:48.286702 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:48.286744 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:48.286889 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.287404 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.288448 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.288669 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.288731 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.288921 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.288955 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.288993 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:48.289082 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 18:51:48.289102 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 18:51:48.289203 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:48.286493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34589 (local address 127.23.218.193:56833)
0809 18:51:48.286550 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:48.286560 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:48.286593 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:48.286782 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:48.286783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:48.286785 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:51:48.286821 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.286823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.287497 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.287499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.288310 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:48.288313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:48.288782 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:48.288783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:48.288837 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:48.288845 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:51:48.289019 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:51:48.289022 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:51:48.289079 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 18:51:48.289127 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 18:51:48.289497 28096 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 18:51:49.810931 27208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:49.811050 27208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:49.811913 27208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76: Sending DeleteTablet for 3 replicas of tablet a9018f1b10724ead996b09b9b4e8ee84
I20260809 18:51:49.812222 27440 tablet_service.cc:1514] Processing DeleteTablet for tablet a9018f1b10724ead996b09b9b4e8ee84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:49 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:37126
I20260809 18:51:49.812222 27673 tablet_service.cc:1514] Processing DeleteTablet for tablet a9018f1b10724ead996b09b9b4e8ee84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:49 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:55262
I20260809 18:51:49.812251 27906 tablet_service.cc:1514] Processing DeleteTablet for tablet a9018f1b10724ead996b09b9b4e8ee84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:49 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:51450
I20260809 18:51:49.812489 28193 tablet_replica.cc:330] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: stopping tablet replica
I20260809 18:51:49.812559 28193 raft_consensus.cc:2229] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:49.812608 28193 raft_consensus.cc:2258] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:49.812847 28193 ts_tablet_manager.cc:1905] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:49.814503 28193 ts_tablet_manager.cc:1918] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 18:51:49.814576 28193 log.cc:1196] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/a9018f1b10724ead996b09b9b4e8ee84
I20260809 18:51:49.814786 28193 ts_tablet_manager.cc:1939] T a9018f1b10724ead996b09b9b4e8ee84 P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting consensus metadata
I20260809 18:51:49.815124 27188 catalog_manager.cc:4841] TS a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093): tablet a9018f1b10724ead996b09b9b4e8ee84 (table test-table [id=eef15c48f98749dab6b45a45088e41c8]) successfully deleted
I20260809 18:51:49.815377 28188 tablet_replica.cc:330] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: stopping tablet replica
I20260809 18:51:49.815446 28188 raft_consensus.cc:2229] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:49.815496 28188 raft_consensus.cc:2258] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:49.815757 28188 ts_tablet_manager.cc:1905] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:49.817463 28188 ts_tablet_manager.cc:1918] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 18:51:49.817529 28188 log.cc:1196] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/a9018f1b10724ead996b09b9b4e8ee84
I20260809 18:51:49.817569 28190 tablet_replica.cc:330] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: stopping tablet replica
I20260809 18:51:49.817646 28190 raft_consensus.cc:2229] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:51:49.817736 28188 ts_tablet_manager.cc:1939] T a9018f1b10724ead996b09b9b4e8ee84 P d8e728dc64844133b2b732a7b36a991a: Deleting consensus metadata
I20260809 18:51:49.817785 28190 raft_consensus.cc:2258] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:49.818020 28190 ts_tablet_manager.cc:1905] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:49.818265 27187 catalog_manager.cc:4841] TS d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519): tablet a9018f1b10724ead996b09b9b4e8ee84 (table test-table [id=eef15c48f98749dab6b45a45088e41c8]) successfully deleted
I20260809 18:51:49.820490 28190 ts_tablet_manager.cc:1918] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 18:51:49.820552 28190 log.cc:1196] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/a9018f1b10724ead996b09b9b4e8ee84
I20260809 18:51:49.820791 28190 ts_tablet_manager.cc:1939] T a9018f1b10724ead996b09b9b4e8ee84 P 51fc1b4f06de43fc829eb37efb336782: Deleting consensus metadata
I20260809 18:51:49.821239 27186 catalog_manager.cc:4841] TS 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589): tablet a9018f1b10724ead996b09b9b4e8ee84 (table test-table [id=eef15c48f98749dab6b45a45088e41c8]) successfully deleted
Aug 09 18:51:49 dist-test-slave-6zbq krb5kdc[27132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301509, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 18:51:49 dist-test-slave-6zbq krb5kdc[27132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301509, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:49.839335 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.830992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44276 (local address 127.23.218.254:39039)
0809 18:51:49.831144 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:49.831147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:49.831158 (+ 11us) server_negotiation.cc:407] Connection header received
0809 18:51:49.831238 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:49.831238 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:49.831250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:49.831319 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:49.831914 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.832453 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.833180 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.833397 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.833453 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.836002 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:49.836004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:49.836004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:49.836014 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:49.837865 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:49.838230 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:49.838232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:49.838232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:49.838294 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:49.838773 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:49.838774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:49.838775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:49.838913 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:49.839009 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:49.839160 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 18:51:49.839243 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260809 18:51:49.841835 27208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:51:49.842033 27208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:51:49.843541 27440 tablet_service.cc:1467] Processing CreateTablet for tablet c5d2c89b959b42f0bc013ef8188e4aac (DEFAULT_TABLE table=test-table [id=b3a5f6cbcb7e4529afe6e936e852cae5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:49.843549 27906 tablet_service.cc:1467] Processing CreateTablet for tablet c5d2c89b959b42f0bc013ef8188e4aac (DEFAULT_TABLE table=test-table [id=b3a5f6cbcb7e4529afe6e936e852cae5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:49.843649 27673 tablet_service.cc:1467] Processing CreateTablet for tablet c5d2c89b959b42f0bc013ef8188e4aac (DEFAULT_TABLE table=test-table [id=b3a5f6cbcb7e4529afe6e936e852cae5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:49.843709 27906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5d2c89b959b42f0bc013ef8188e4aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:49.843711 27440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5d2c89b959b42f0bc013ef8188e4aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:49.843763 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5d2c89b959b42f0bc013ef8188e4aac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:49.844929 28211 tablet_bootstrap.cc:492] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: Bootstrap starting.
I20260809 18:51:49.845183 28212 tablet_bootstrap.cc:492] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: Bootstrap starting.
I20260809 18:51:49.845336 28215 tablet_bootstrap.cc:492] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: Bootstrap starting.
I20260809 18:51:49.845419 28211 tablet_bootstrap.cc:654] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:49.845701 28215 tablet_bootstrap.cc:654] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:49.845729 28212 tablet_bootstrap.cc:654] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:49.846117 28211 tablet_bootstrap.cc:492] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: No bootstrap required, opened a new log
I20260809 18:51:49.846176 28211 ts_tablet_manager.cc:1397] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:49.846282 28212 tablet_bootstrap.cc:492] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: No bootstrap required, opened a new log
I20260809 18:51:49.846333 28212 ts_tablet_manager.cc:1397] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:49.846297 28211 raft_consensus.cc:348] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.846421 28211 raft_consensus.cc:374] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:49.846447 28211 raft_consensus.cc:729] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53dbd27a4494ba1a0b16c36f50c76e7, State: Initialized, Role: FOLLOWER
I20260809 18:51:49.846483 28212 raft_consensus.cc:348] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.846516 28211 consensus_queue.cc:260] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.846554 28212 raft_consensus.cc:374] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:49.846575 28212 raft_consensus.cc:729] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e728dc64844133b2b732a7b36a991a, State: Initialized, Role: FOLLOWER
I20260809 18:51:49.846622 28212 consensus_queue.cc:260] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.846707 28211 ts_tablet_manager.cc:1428] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:49.846717 28212 ts_tablet_manager.cc:1428] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:49.847224 28215 tablet_bootstrap.cc:492] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: No bootstrap required, opened a new log
I20260809 18:51:49.847285 28215 ts_tablet_manager.cc:1397] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:51:49.847394 28215 raft_consensus.cc:348] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.847462 28215 raft_consensus.cc:374] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:49.847486 28215 raft_consensus.cc:729] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51fc1b4f06de43fc829eb37efb336782, State: Initialized, Role: FOLLOWER
I20260809 18:51:49.847525 28215 consensus_queue.cc:260] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.847620 28215 ts_tablet_manager.cc:1428] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:49.866115 28149 raft_consensus.cc:482] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:49.866232 28149 raft_consensus.cc:504] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.866434 28149 leader_election.cc:288] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519), a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093)
I20260809 18:51:49.866691 27936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5d2c89b959b42f0bc013ef8188e4aac" candidate_uuid: "51fc1b4f06de43fc829eb37efb336782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" is_pre_election: true
I20260809 18:51:49.866690 27471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5d2c89b959b42f0bc013ef8188e4aac" candidate_uuid: "51fc1b4f06de43fc829eb37efb336782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e728dc64844133b2b732a7b36a991a" is_pre_election: true
I20260809 18:51:49.866793 27936 raft_consensus.cc:2454] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51fc1b4f06de43fc829eb37efb336782 in term 0.
I20260809 18:51:49.866803 27471 raft_consensus.cc:2454] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51fc1b4f06de43fc829eb37efb336782 in term 0.
I20260809 18:51:49.866954 27590 leader_election.cc:302] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fc1b4f06de43fc829eb37efb336782, a53dbd27a4494ba1a0b16c36f50c76e7; no voters:
I20260809 18:51:49.867057 28149 raft_consensus.cc:2781] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:51:49.867103 28149 raft_consensus.cc:482] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:51:49.867121 28149 raft_consensus.cc:3037] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:49.867957 28149 raft_consensus.cc:504] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.868088 28149 leader_election.cc:288] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [CANDIDATE]: Term 1 election: Requested vote from peers d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519), a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093)
I20260809 18:51:49.868249 27936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5d2c89b959b42f0bc013ef8188e4aac" candidate_uuid: "51fc1b4f06de43fc829eb37efb336782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7"
I20260809 18:51:49.868259 27471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5d2c89b959b42f0bc013ef8188e4aac" candidate_uuid: "51fc1b4f06de43fc829eb37efb336782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e728dc64844133b2b732a7b36a991a"
I20260809 18:51:49.868330 27471 raft_consensus.cc:3037] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:49.868331 27936 raft_consensus.cc:3037] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:49.868973 27936 raft_consensus.cc:2454] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51fc1b4f06de43fc829eb37efb336782 in term 1.
I20260809 18:51:49.869290 27590 leader_election.cc:302] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fc1b4f06de43fc829eb37efb336782, a53dbd27a4494ba1a0b16c36f50c76e7; no voters:
I20260809 18:51:49.869442 28149 raft_consensus.cc:2781] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:49.869585 28149 raft_consensus.cc:686] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 1 LEADER]: Becoming Leader. State: Replica: 51fc1b4f06de43fc829eb37efb336782, State: Running, Role: LEADER
I20260809 18:51:49.869690 27471 raft_consensus.cc:2454] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51fc1b4f06de43fc829eb37efb336782 in term 1.
I20260809 18:51:49.869664 28149 consensus_queue.cc:237] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.870635 27209 catalog_manager.cc:5495] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194). New cstate: current_term: 1 leader_uuid: "51fc1b4f06de43fc829eb37efb336782" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } health_report { overall_health: UNKNOWN } } }
I20260809 18:51:49.881438 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.878609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.23.218.194:34589)
0809 18:51:49.878775 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:49.878777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:49.878789 (+ 12us) server_negotiation.cc:407] Connection header received
0809 18:51:49.878808 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:49.878808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:49.878819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:49.878875 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:49.879009 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.879925 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.880750 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.880966 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.881024 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.881170 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:49.881207 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:49.881247 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:49.881323 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 18:51:49.881348 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260809 18:51:49.884572 27936 raft_consensus.cc:1261] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Refusing update from remote peer 51fc1b4f06de43fc829eb37efb336782: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:49.884660 27471 raft_consensus.cc:1261] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Refusing update from remote peer 51fc1b4f06de43fc829eb37efb336782: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:49.884951 28169 consensus_queue.cc:1035] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:49.885105 28149 consensus_queue.cc:1035] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:49.897459 27206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:49.897626 27206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:49.898556 27206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76: Sending DeleteTablet for 3 replicas of tablet c5d2c89b959b42f0bc013ef8188e4aac
I20260809 18:51:49.898921 27906 tablet_service.cc:1514] Processing DeleteTablet for tablet c5d2c89b959b42f0bc013ef8188e4aac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:49 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:51450
I20260809 18:51:49.899003 27673 tablet_service.cc:1514] Processing DeleteTablet for tablet c5d2c89b959b42f0bc013ef8188e4aac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:49 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:55262
I20260809 18:51:49.899040 28193 tablet_replica.cc:330] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: stopping tablet replica
I20260809 18:51:49.899057 27440 tablet_service.cc:1514] Processing DeleteTablet for tablet c5d2c89b959b42f0bc013ef8188e4aac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:49 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:37126
I20260809 18:51:49.899089 28193 raft_consensus.cc:2229] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:49.899168 28193 raft_consensus.cc:2258] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:49.899201 28188 tablet_replica.cc:330] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: stopping tablet replica
I20260809 18:51:49.899243 28190 tablet_replica.cc:330] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: stopping tablet replica
I20260809 18:51:49.899277 28188 raft_consensus.cc:2229] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:49.899302 28190 raft_consensus.cc:2229] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:51:49.899340 28188 raft_consensus.cc:2258] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:49.899374 28190 raft_consensus.cc:2258] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:49.899371 27206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:51:49.899591 27206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:51:49.899713 28190 ts_tablet_manager.cc:1905] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:49.899886 28193 ts_tablet_manager.cc:1905] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:49.901316 27673 tablet_service.cc:1467] Processing CreateTablet for tablet affecdff615b42c2b0302b5c49414ed3 (DEFAULT_TABLE table=test-table [id=78319f484d4a4f6a8d180fe4cdab97bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:49.901340 27906 tablet_service.cc:1467] Processing CreateTablet for tablet affecdff615b42c2b0302b5c49414ed3 (DEFAULT_TABLE table=test-table [id=78319f484d4a4f6a8d180fe4cdab97bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:49.901496 27906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet affecdff615b42c2b0302b5c49414ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:49.902009 28193 ts_tablet_manager.cc:1918] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:51:49.902243 28193 log.cc:1196] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/c5d2c89b959b42f0bc013ef8188e4aac
I20260809 18:51:49.902554 28193 ts_tablet_manager.cc:1939] T c5d2c89b959b42f0bc013ef8188e4aac P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting consensus metadata
I20260809 18:51:49.902789 28190 ts_tablet_manager.cc:1918] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:51:49.902801 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet affecdff615b42c2b0302b5c49414ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:49.902860 28190 log.cc:1196] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/c5d2c89b959b42f0bc013ef8188e4aac
I20260809 18:51:49.903023 28211 tablet_bootstrap.cc:492] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: Bootstrap starting.
I20260809 18:51:49.903054 28190 ts_tablet_manager.cc:1939] T c5d2c89b959b42f0bc013ef8188e4aac P 51fc1b4f06de43fc829eb37efb336782: Deleting consensus metadata
I20260809 18:51:49.903131 27188 catalog_manager.cc:4841] TS a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093): tablet c5d2c89b959b42f0bc013ef8188e4aac (table test-table [id=b3a5f6cbcb7e4529afe6e936e852cae5]) successfully deleted
I20260809 18:51:49.903331 27440 tablet_service.cc:1467] Processing CreateTablet for tablet affecdff615b42c2b0302b5c49414ed3 (DEFAULT_TABLE table=test-table [id=78319f484d4a4f6a8d180fe4cdab97bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:51:49.903453 27440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet affecdff615b42c2b0302b5c49414ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:49.903473 28211 tablet_bootstrap.cc:654] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:49.903985 28215 tablet_bootstrap.cc:492] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: Bootstrap starting.
I20260809 18:51:49.904134 28211 tablet_bootstrap.cc:492] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: No bootstrap required, opened a new log
I20260809 18:51:49.904181 28211 ts_tablet_manager.cc:1397] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:49.904237 28215 tablet_bootstrap.cc:654] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:49.904342 28211 raft_consensus.cc:348] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.904415 28211 raft_consensus.cc:374] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:49.904431 28211 raft_consensus.cc:729] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53dbd27a4494ba1a0b16c36f50c76e7, State: Initialized, Role: FOLLOWER
I20260809 18:51:49.904474 28211 consensus_queue.cc:260] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.904559 28211 ts_tablet_manager.cc:1428] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:49.904610 28188 ts_tablet_manager.cc:1905] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:49.904934 28212 tablet_bootstrap.cc:492] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a: Bootstrap starting.
I20260809 18:51:49.905287 28212 tablet_bootstrap.cc:654] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:49.905803 28212 tablet_bootstrap.cc:492] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a: No bootstrap required, opened a new log
I20260809 18:51:49.905848 28212 ts_tablet_manager.cc:1397] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 18:51:49.905972 28188 ts_tablet_manager.cc:1918] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:51:49.906038 28188 log.cc:1196] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/c5d2c89b959b42f0bc013ef8188e4aac
I20260809 18:51:49.905972 28212 raft_consensus.cc:348] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.906075 28212 raft_consensus.cc:374] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:49.906196 28212 raft_consensus.cc:729] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e728dc64844133b2b732a7b36a991a, State: Initialized, Role: FOLLOWER
I20260809 18:51:49.906267 28212 consensus_queue.cc:260] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.906380 28188 ts_tablet_manager.cc:1939] T c5d2c89b959b42f0bc013ef8188e4aac P d8e728dc64844133b2b732a7b36a991a: Deleting consensus metadata
I20260809 18:51:49.906466 28212 ts_tablet_manager.cc:1428] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 18:51:49.906754 28215 tablet_bootstrap.cc:492] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: No bootstrap required, opened a new log
I20260809 18:51:49.906790 28215 ts_tablet_manager.cc:1397] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260809 18:51:49.906896 28215 raft_consensus.cc:348] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.906957 28215 raft_consensus.cc:374] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:49.906976 28215 raft_consensus.cc:729] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51fc1b4f06de43fc829eb37efb336782, State: Initialized, Role: FOLLOWER
I20260809 18:51:49.907020 28215 consensus_queue.cc:260] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.907029 27187 catalog_manager.cc:4841] TS d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519): tablet c5d2c89b959b42f0bc013ef8188e4aac (table test-table [id=b3a5f6cbcb7e4529afe6e936e852cae5]) successfully deleted
I20260809 18:51:49.907107 28215 ts_tablet_manager.cc:1428] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:49.908281 27186 catalog_manager.cc:4841] TS 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589): tablet c5d2c89b959b42f0bc013ef8188e4aac (table test-table [id=b3a5f6cbcb7e4529afe6e936e852cae5]) successfully deleted
I20260809 18:51:49.927591 28221 raft_consensus.cc:482] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:51:49.927796 28221 raft_consensus.cc:504] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.928467 28221 leader_election.cc:288] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519), 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589)
I20260809 18:51:49.933573 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.928847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:47677 (local address 127.23.218.194:34589)
0809 18:51:49.928928 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:49.928930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:49.929114 (+ 184us) server_negotiation.cc:407] Connection header received
0809 18:51:49.929347 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:49.929350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:49.929369 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:49.929546 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:49.929769 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.930862 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.932554 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.932991 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.933131 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.933142 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:49.933409 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 18:51:49.933478 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 18:51:49.933614 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.928791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:49357 (local address 127.23.218.193:41519)
0809 18:51:49.929057 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:49.929059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:49.929280 (+ 221us) server_negotiation.cc:407] Connection header received
0809 18:51:49.929351 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:49.929352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:49.929367 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:49.929526 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:51:49.929776 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.930818 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.932548 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.933050 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.933175 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.933194 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:49.933412 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 18:51:49.933489 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":157,"threads_started":1}
I20260809 18:51:49.933923 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.928694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41519 (local address 127.23.218.195:49357)
0809 18:51:49.929063 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:49.929073 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:49.929186 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:49.929600 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:49.929601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:49.929602 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:49.929689 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:49.929690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.931059 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.931066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:49.932355 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:49.932364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.933199 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.933201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:49.933304 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.933309 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:51:49.933396 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 18:51:49.933467 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":160,"thread_start_us":115,"threads_started":1}
I20260809 18:51:49.933979 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.928691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34589 (local address 127.23.218.195:47677)
0809 18:51:49.928946 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:49.928977 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:49.929165 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:49.929573 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:49.929577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:49.929584 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:51:49.929676 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:49.929678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.931137 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.931138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:49.932355 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:49.932363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.933149 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.933152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:49.933291 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.933295 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:51:49.933378 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 18:51:49.933443 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":30,"thread_start_us":282,"threads_started":1}
I20260809 18:51:49.934314 27471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "affecdff615b42c2b0302b5c49414ed3" candidate_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e728dc64844133b2b732a7b36a991a" is_pre_election: true
I20260809 18:51:49.934393 27702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "affecdff615b42c2b0302b5c49414ed3" candidate_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fc1b4f06de43fc829eb37efb336782" is_pre_election: true
I20260809 18:51:49.934533 27471 raft_consensus.cc:2454] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53dbd27a4494ba1a0b16c36f50c76e7 in term 0.
I20260809 18:51:49.934536 27702 raft_consensus.cc:2454] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53dbd27a4494ba1a0b16c36f50c76e7 in term 0.
I20260809 18:51:49.934834 27826 leader_election.cc:302] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51fc1b4f06de43fc829eb37efb336782, a53dbd27a4494ba1a0b16c36f50c76e7; no voters:
I20260809 18:51:49.935019 28221 raft_consensus.cc:2781] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:51:49.935081 28221 raft_consensus.cc:482] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:51:49.935097 28221 raft_consensus.cc:3037] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:49.937237 28221 raft_consensus.cc:504] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.937922 28221 leader_election.cc:288] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [CANDIDATE]: Term 1 election: Requested vote from peers d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519), 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589)
I20260809 18:51:49.938174 27702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "affecdff615b42c2b0302b5c49414ed3" candidate_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51fc1b4f06de43fc829eb37efb336782"
I20260809 18:51:49.938181 27471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "affecdff615b42c2b0302b5c49414ed3" candidate_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e728dc64844133b2b732a7b36a991a"
I20260809 18:51:49.938292 27471 raft_consensus.cc:3037] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:49.938292 27702 raft_consensus.cc:3037] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:49.939219 27471 raft_consensus.cc:2454] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53dbd27a4494ba1a0b16c36f50c76e7 in term 1.
I20260809 18:51:49.939229 27702 raft_consensus.cc:2454] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53dbd27a4494ba1a0b16c36f50c76e7 in term 1.
I20260809 18:51:49.939535 27830 leader_election.cc:302] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53dbd27a4494ba1a0b16c36f50c76e7, d8e728dc64844133b2b732a7b36a991a; no voters:
I20260809 18:51:49.940384 28221 raft_consensus.cc:2781] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:49.940727 28221 raft_consensus.cc:686] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 LEADER]: Becoming Leader. State: Replica: a53dbd27a4494ba1a0b16c36f50c76e7, State: Running, Role: LEADER
I20260809 18:51:49.940858 28221 consensus_queue.cc:237] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } }
I20260809 18:51:49.942893 27208 catalog_manager.cc:5495] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195). New cstate: current_term: 1 leader_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a53dbd27a4494ba1a0b16c36f50c76e7" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 36093 } health_report { overall_health: HEALTHY } } }
I20260809 18:51:49.954501 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.952436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.23.218.195:36093)
0809 18:51:49.952568 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:49.952570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:49.952578 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:51:49.952612 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:49.952613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:49.952630 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:49.952703 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:49.952784 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.953247 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.953899 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.954081 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.954122 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.954213 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:49.954314 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:49.954393 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:49.954429 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 18:51:49.954444 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260809 18:51:49.958743 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.956729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:42629 (local address 127.23.218.193:41519)
0809 18:51:49.956780 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:49.956782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:49.956865 (+ 83us) server_negotiation.cc:407] Connection header received
0809 18:51:49.956903 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:49.956904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:49.956911 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:49.956953 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:49.957062 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.957518 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.958176 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.958402 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.958450 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.958588 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:49.958611 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:49.958642 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:49.958684 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 18:51:49.958699 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 18:51:49.958760 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.956611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41519 (local address 127.23.218.195:42629)
0809 18:51:49.956811 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:49.956820 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:49.956869 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:49.956967 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:49.956967 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:49.956968 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:51:49.957023 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:49.957024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.957571 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.957572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:49.958108 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:49.958109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.958472 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.958473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:49.958517 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.958531 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:51:49.958643 (+ 112us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:51:49.958644 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 18:51:49.958669 (+ 25us) client_negotiation.cc:240] Negotiation successful
0809 18:51:49.958698 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":70,"threads_started":1}
I20260809 18:51:49.960565 27471 raft_consensus.cc:1261] T affecdff615b42c2b0302b5c49414ed3 P d8e728dc64844133b2b732a7b36a991a [term 1 FOLLOWER]: Refusing update from remote peer a53dbd27a4494ba1a0b16c36f50c76e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:49.960572 27702 raft_consensus.cc:1261] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Refusing update from remote peer a53dbd27a4494ba1a0b16c36f50c76e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:51:49.960809 28221 consensus_queue.cc:1035] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8e728dc64844133b2b732a7b36a991a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:49.960942 28243 consensus_queue.cc:1035] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51fc1b4f06de43fc829eb37efb336782" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:51:49.967550 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.965498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:47721 (local address 127.23.218.195:36093)
0809 18:51:49.965539 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:49.965540 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:49.965548 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:51:49.965611 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:49.965612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:49.965621 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:49.965668 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:51:49.965763 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.966192 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.966978 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.967269 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.967316 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.967417 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:51:49.967438 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:51:49.967467 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:49.967497 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 18:51:49.967511 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:51:49.967574 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:49.965376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:36093 (local address 127.23.218.193:47721)
0809 18:51:49.965501 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:49.965511 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:49.965556 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:49.965672 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:49.965672 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:49.965674 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:51:49.965719 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:49.965721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.966256 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.966257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:49.966896 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:49.966899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:49.967332 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:49.967333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:49.967376 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:49.967380 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:51:49.967468 (+ 88us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:51:49.967468 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 18:51:49.967494 (+ 26us) client_negotiation.cc:240] Negotiation successful
0809 18:51:49.967525 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":80,"threads_started":1}
W20260809 18:51:49.967867 28158 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 18:51:51.485231 27206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:51.485337 27206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:51:51.486105 27206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0857a726cc84d02820b23ddb3d8cf76: Sending DeleteTablet for 3 replicas of tablet affecdff615b42c2b0302b5c49414ed3
I20260809 18:51:51.486332 27440 tablet_service.cc:1514] Processing DeleteTablet for tablet affecdff615b42c2b0302b5c49414ed3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:51 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:37126
I20260809 18:51:51.486362 27906 tablet_service.cc:1514] Processing DeleteTablet for tablet affecdff615b42c2b0302b5c49414ed3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:51 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:51450
I20260809 18:51:51.486533 27673 tablet_service.cc:1514] Processing DeleteTablet for tablet affecdff615b42c2b0302b5c49414ed3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:51:51 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:55262
I20260809 18:51:51.486701 28294 tablet_replica.cc:330] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: stopping tablet replica
I20260809 18:51:51.486759 28294 raft_consensus.cc:2229] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:51:51.486802 28294 raft_consensus.cc:2258] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:51.486805 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27336
I20260809 18:51:51.496775 28294 ts_tablet_manager.cc:1905] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:51.498100 28294 ts_tablet_manager.cc:1918] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 18:51:51.498140 28294 log.cc:1196] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/affecdff615b42c2b0302b5c49414ed3
I20260809 18:51:51.498296 28294 ts_tablet_manager.cc:1939] T affecdff615b42c2b0302b5c49414ed3 P 51fc1b4f06de43fc829eb37efb336782: Deleting consensus metadata
I20260809 18:51:51.498385 28291 tablet_replica.cc:330] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: stopping tablet replica
I20260809 18:51:51.498441 28291 raft_consensus.cc:2229] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:51:51.498529 28291 raft_consensus.cc:2258] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:51:51.498860 28291 ts_tablet_manager.cc:1905] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:51:51.499130 27186 catalog_manager.cc:4841] TS 51fc1b4f06de43fc829eb37efb336782 (127.23.218.194:34589): tablet affecdff615b42c2b0302b5c49414ed3 (table test-table [id=78319f484d4a4f6a8d180fe4cdab97bf]) successfully deleted
I20260809 18:51:51.500195 28291 ts_tablet_manager.cc:1918] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 18:51:51.500243 28291 log.cc:1196] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/affecdff615b42c2b0302b5c49414ed3
I20260809 18:51:51.500437 28291 ts_tablet_manager.cc:1939] T affecdff615b42c2b0302b5c49414ed3 P a53dbd27a4494ba1a0b16c36f50c76e7: Deleting consensus metadata
I20260809 18:51:51.500782 27188 catalog_manager.cc:4841] TS a53dbd27a4494ba1a0b16c36f50c76e7 (127.23.218.195:36093): tablet affecdff615b42c2b0302b5c49414ed3 (table test-table [id=78319f484d4a4f6a8d180fe4cdab97bf]) successfully deleted
W20260809 18:51:51.505296 27187 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:41519 (error 108)
I20260809 18:51:51.505494 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27567
I20260809 18:51:51.516717 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:51.505683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41519 (local address 127.0.0.1:37162)
0809 18:51:51.516635 (+ 10952us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:51.516683 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:41519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10928,"thread_start_us":62,"threads_started":1}
W20260809 18:51:51.516844 27187 catalog_manager.cc:4584] TS d8e728dc64844133b2b732a7b36a991a (127.23.218.193:41519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet affecdff615b42c2b0302b5c49414ed3: Network error: Client connection negotiation failed: client connection to 127.23.218.193:41519: connect: Connection refused (error 111)
I20260809 18:51:51.521397 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27800
I20260809 18:51:51.533170 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27154
2026-08-09T18:51:51Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5555 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:51:51 dist-test-slave-6zbq krb5kdc[28303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:51:51 dist-test-slave-6zbq krb5kdc[28303](info): set up 2 sockets
Aug 09 18:51:51 dist-test-slave-6zbq krb5kdc[28303](info): commencing operation
krb5kdc: starting...
W20260809 18:51:52.664443 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:51:52 dist-test-slave-6zbq krb5kdc[28303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:51:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:51:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:51:52.768607 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34693
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:34693
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:52.845852 28319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:52.846020 28319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:52.846041 28319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:52.847106 28319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:51:52.847138 28319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:52.847151 28319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:52.847162 28319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:51:52.847177 28319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:51:52.848527 28319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:34693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:52.848694 28319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:52.848861 28319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 18:51:52.853222 28319 server_base.cc:1034] running on GCE node
W20260809 18:51:52.853257 28329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:52.853188 28325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:52.853195 28326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:52.853622 28319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:52.853886 28319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:52.855015 28319 hybrid_clock.cc:648] HybridClock initialized: now 1786301512855000 us; error 31 us; skew 500 ppm
Aug 09 18:51:52 dist-test-slave-6zbq krb5kdc[28303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301512, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:52.861495 28319 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:51:52.861920 28319 webserver.cc:460] Webserver started at http://127.23.218.254:45537/ using document root <none> and password file <none>
I20260809 18:51:52.862151 28319 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:52.862200 28319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:52.862318 28319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:52.863324 28319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "492793bc03494028b186add63df992b3"
format_stamp: "Formatted at 2026-08-09 18:51:52 on dist-test-slave-6zbq"
server_key: "4d1b8d0ed093d9f47fbbedfa6782d1aa"
server_key_iv: "023d1b51264c662bef8852b769b53430"
server_key_version: "encryptionkey@0"
I20260809 18:51:52.863649 28319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "492793bc03494028b186add63df992b3"
format_stamp: "Formatted at 2026-08-09 18:51:52 on dist-test-slave-6zbq"
server_key: "4d1b8d0ed093d9f47fbbedfa6782d1aa"
server_key_iv: "023d1b51264c662bef8852b769b53430"
server_key_version: "encryptionkey@0"
I20260809 18:51:52.864818 28319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:51:52.865470 28342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:52.865631 28319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:52.865720 28319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "492793bc03494028b186add63df992b3"
format_stamp: "Formatted at 2026-08-09 18:51:52 on dist-test-slave-6zbq"
server_key: "4d1b8d0ed093d9f47fbbedfa6782d1aa"
server_key_iv: "023d1b51264c662bef8852b769b53430"
server_key_version: "encryptionkey@0"
I20260809 18:51:52.865778 28319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:52.886538 28319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:52.888216 28319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:52.888345 28319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:52.891964 28319 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:34693
I20260809 18:51:52.891999 28428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:34693 every 8 connection(s)
I20260809 18:51:52.892285 28319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:51:52.892494 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 28319
I20260809 18:51:52.892576 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:51:52.892750 24427 external_mini_cluster.cc:1428] Setting key 6731a724fab9f3de5591c7d04da8fb80
I20260809 18:51:52.893110 28429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:52.895697 28429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3: Bootstrap starting.
I20260809 18:51:52.896245 28429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:52.896495 28429 log.cc:824] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:52.897099 28429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3: No bootstrap required, opened a new log
Aug 09 18:51:52 dist-test-slave-6zbq krb5kdc[28303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:52.898973 28429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492793bc03494028b186add63df992b3" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34693 } }
I20260809 18:51:52.899127 28429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:52.899151 28429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492793bc03494028b186add63df992b3, State: Initialized, Role: FOLLOWER
I20260809 18:51:52.899262 28429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492793bc03494028b186add63df992b3" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34693 } }
I20260809 18:51:52.899341 28429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:51:52.899380 28429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:51:52.899427 28429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:52.899999 28429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492793bc03494028b186add63df992b3" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34693 } }
I20260809 18:51:52.900089 28429 leader_election.cc:302] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 492793bc03494028b186add63df992b3; no voters:
I20260809 18:51:52.900260 28429 leader_election.cc:288] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:51:52.900319 28437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:52.900454 28437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [term 1 LEADER]: Becoming Leader. State: Replica: 492793bc03494028b186add63df992b3, State: Running, Role: LEADER
I20260809 18:51:52.900506 28429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:51:52.900616 28437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492793bc03494028b186add63df992b3" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34693 } }
I20260809 18:51:52.901834 28439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 492793bc03494028b186add63df992b3. Latest consensus state: current_term: 1 leader_uuid: "492793bc03494028b186add63df992b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492793bc03494028b186add63df992b3" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34693 } } }
I20260809 18:51:52.901918 28439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:52.901996 28440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "492793bc03494028b186add63df992b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492793bc03494028b186add63df992b3" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34693 } } }
I20260809 18:51:52.902124 28440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3 [sys.catalog]: This master's current role is: LEADER
W20260809 18:51:52.902547 28459 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:51:52.902607 28459 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:51:52.902669 28460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:51:52.902839 28460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:51:52.902885 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:52.893237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43840 (local address 127.23.218.254:34693)
0809 18:51:52.893411 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:52.893419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:52.893442 (+ 23us) server_negotiation.cc:407] Connection header received
0809 18:51:52.893844 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:52.893850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:52.893891 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:52.894007 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:52.894662 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:52.895499 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:52.896235 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:52.896694 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:52.896788 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:52.899056 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:52.899060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:52.899060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:52.899070 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:52.900816 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:52.901271 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:52.901273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:52.901274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:52.901358 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:52.901572 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:52.901572 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:52.901573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:52.901876 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:52.902491 (+ 615us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:52.902564 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 18:51:52.902703 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260809 18:51:52.904271 28460 catalog_manager.cc:1347] Generated new cluster ID: 5b6e7bde345443a99f8fe4576e71cc52
I20260809 18:51:52.904323 28460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:51:52.912523 28460 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:51:52.913214 28460 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:51:52.932750 28460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 492793bc03494028b186add63df992b3: Generated new TSK 0
I20260809 18:51:52.932941 28460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:51:52.963218 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34693
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:53.042064 28469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:53.042255 28469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:53.042276 28469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:53.043408 28469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:53.043449 28469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:53.043473 28469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:51:53.044852 28469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:53.045045 28469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:53.045254 28469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:53.049839 28478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:53.049862 28479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:53.049865 28482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:53.049926 28469 server_base.cc:1034] running on GCE node
I20260809 18:51:53.050206 28469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:53.050457 28469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:53.051599 28469 hybrid_clock.cc:648] HybridClock initialized: now 1786301513051584 us; error 31 us; skew 500 ppm
Aug 09 18:51:53 dist-test-slave-6zbq krb5kdc[28303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301513, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:53.053803 28469 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:51:53.054177 28469 webserver.cc:460] Webserver started at http://127.23.218.193:42609/ using document root <none> and password file <none>
I20260809 18:51:53.054396 28469 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:53.054448 28469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:53.054574 28469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:53.055533 28469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "7ccd261e7e09446480948487b1f03954"
format_stamp: "Formatted at 2026-08-09 18:51:53 on dist-test-slave-6zbq"
server_key: "df4003ecb1b27151e207315707cc18f9"
server_key_iv: "8d6184ba4cacc7fccb6c061ef4913fd0"
server_key_version: "encryptionkey@0"
I20260809 18:51:53.055868 28469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "7ccd261e7e09446480948487b1f03954"
format_stamp: "Formatted at 2026-08-09 18:51:53 on dist-test-slave-6zbq"
server_key: "df4003ecb1b27151e207315707cc18f9"
server_key_iv: "8d6184ba4cacc7fccb6c061ef4913fd0"
server_key_version: "encryptionkey@0"
I20260809 18:51:53.057046 28469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:51:53.057687 28497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.057835 28469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.057916 28469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "7ccd261e7e09446480948487b1f03954"
format_stamp: "Formatted at 2026-08-09 18:51:53 on dist-test-slave-6zbq"
server_key: "df4003ecb1b27151e207315707cc18f9"
server_key_iv: "8d6184ba4cacc7fccb6c061ef4913fd0"
server_key_version: "encryptionkey@0"
I20260809 18:51:53.057982 28469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:53.083472 28469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:53.084995 28469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:53.085117 28469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:53.085309 28469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:53.085577 28469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:53.085610 28469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.085646 28469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:53.085667 28469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.091032 28469 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:34477
I20260809 18:51:53.091065 28689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:34477 every 8 connection(s)
I20260809 18:51:53.091334 28469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
Aug 09 18:51:53 dist-test-slave-6zbq krb5kdc[28303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301513, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:53.097869 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 28469
I20260809 18:51:53.097939 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:51:53.098075 24427 external_mini_cluster.cc:1428] Setting key f56a29c69b985b7bc82d1b7d2de632d3
I20260809 18:51:53.099342 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:53.091916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:38167 (local address 127.23.218.254:34693)
0809 18:51:53.091954 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:53.091956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:53.092157 (+ 201us) server_negotiation.cc:407] Connection header received
0809 18:51:53.092568 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:53.092568 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:53.092577 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:53.092634 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:53.093436 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.093918 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.094847 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.095035 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.095084 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:53.096887 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:53.096888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:53.096888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:53.096894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:53.097978 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:53.098435 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:53.098436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:53.098436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:53.098488 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:53.098744 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:53.098744 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:53.098745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:53.098875 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:53.098940 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:53.099155 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 18:51:53.099257 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 18:51:53.099386 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:53.091889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34693 (local address 127.23.218.193:38167)
0809 18:51:53.092046 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:53.092082 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:53.092501 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:53.092676 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:53.092679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:53.093193 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:53.093372 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:53.093376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.093975 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.093976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:53.094787 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:53.094790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.095090 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.095091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:53.095181 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:53.095607 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:53.095615 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:53.096818 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:53.098051 (+ 1233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:53.098058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:53.098060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:53.098371 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:53.098545 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:53.098546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:53.098546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:53.098678 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:53.098939 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:53.098943 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:53.099046 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 18:51:53.099120 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 18:51:53.099236 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260809 18:51:53.100015 28691 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34693
I20260809 18:51:53.100116 28691 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:53.100302 28691 heartbeater.cc:511] Master 127.23.218.254:34693 requested a full tablet report, sending...
I20260809 18:51:53.100790 28367 ts_manager.cc:194] Registered new tserver with Master: 7ccd261e7e09446480948487b1f03954 (127.23.218.193:34477)
I20260809 18:51:53.101527 28367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:38167
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:51:53.120528 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34693
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:53.200142 28699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:53.200320 28699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:53.200340 28699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:53.201540 28699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:53.201577 28699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:53.201610 28699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:51:53.203088 28699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:53.203266 28699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:53.203428 28699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:53.208194 28710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:53.208192 28713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:53.208204 28709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:53.208448 28699 server_base.cc:1034] running on GCE node
I20260809 18:51:53.208654 28699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:53.208858 28699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:53.210002 28699 hybrid_clock.cc:648] HybridClock initialized: now 1786301513209985 us; error 28 us; skew 500 ppm
Aug 09 18:51:53 dist-test-slave-6zbq krb5kdc[28303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301513, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:53.212447 28699 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:51:53.212863 28699 webserver.cc:460] Webserver started at http://127.23.218.194:39049/ using document root <none> and password file <none>
I20260809 18:51:53.213035 28699 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:53.213073 28699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:53.213163 28699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:53.214113 28699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "0c7a71aac86e440bb5741210b9d27568"
format_stamp: "Formatted at 2026-08-09 18:51:53 on dist-test-slave-6zbq"
server_key: "6174950430b475f0a99a20e49d73afdd"
server_key_iv: "42d9411a920ea2b9cfa4062da47f1973"
server_key_version: "encryptionkey@0"
I20260809 18:51:53.214463 28699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "0c7a71aac86e440bb5741210b9d27568"
format_stamp: "Formatted at 2026-08-09 18:51:53 on dist-test-slave-6zbq"
server_key: "6174950430b475f0a99a20e49d73afdd"
server_key_iv: "42d9411a920ea2b9cfa4062da47f1973"
server_key_version: "encryptionkey@0"
I20260809 18:51:53.215620 28699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:51:53.216228 28724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.216372 28699 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:53.216454 28699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "0c7a71aac86e440bb5741210b9d27568"
format_stamp: "Formatted at 2026-08-09 18:51:53 on dist-test-slave-6zbq"
server_key: "6174950430b475f0a99a20e49d73afdd"
server_key_iv: "42d9411a920ea2b9cfa4062da47f1973"
server_key_version: "encryptionkey@0"
I20260809 18:51:53.216507 28699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:53.229586 28699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:53.231285 28699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:53.231387 28699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:53.231549 28699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:53.231818 28699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:53.231851 28699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.231871 28699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:53.231894 28699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.237196 28699 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:34241
I20260809 18:51:53.237246 28913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:34241 every 8 connection(s)
I20260809 18:51:53.237512 28699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
Aug 09 18:51:53 dist-test-slave-6zbq krb5kdc[28303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301513, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:53.244796 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 28699
I20260809 18:51:53.244895 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:51:53.245064 24427 external_mini_cluster.cc:1428] Setting key 4b5ebf2e1a9e5fda83b00aceb75985f7
I20260809 18:51:53.245604 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:53.238133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:56343 (local address 127.23.218.254:34693)
0809 18:51:53.238192 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:53.238194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:53.238394 (+ 200us) server_negotiation.cc:407] Connection header received
0809 18:51:53.238863 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:53.238865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:53.238875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:53.238926 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:53.239735 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.240215 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.241122 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.241308 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.241352 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:53.243164 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:53.243166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:53.243166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:53.243173 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:53.244376 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:53.244675 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:53.244676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:53.244676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:53.244720 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:53.244985 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:53.244986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:53.244987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:53.245096 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:53.245179 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:53.245432 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 18:51:53.245517 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 18:51:53.245755 28919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:53.238093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34693 (local address 127.23.218.194:56343)
0809 18:51:53.238272 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:53.238310 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:53.238787 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:53.238977 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:53.238980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:53.239496 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:53.239672 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:53.239677 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.240291 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.240291 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:53.241041 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:53.241043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.241360 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.241361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:53.241449 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:53.241804 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:53.241811 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:53.243077 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:53.244436 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:53.244439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:53.244440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:53.244619 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:53.244773 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:53.244774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:53.244774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:53.244901 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:53.245173 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:53.245181 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:53.245309 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 18:51:53.245396 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 18:51:53.245540 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260809 18:51:53.246685 28914 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34693
I20260809 18:51:53.246768 28914 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:53.246927 28914 heartbeater.cc:511] Master 127.23.218.254:34693 requested a full tablet report, sending...
I20260809 18:51:53.247272 28367 ts_manager.cc:194] Registered new tserver with Master: 0c7a71aac86e440bb5741210b9d27568 (127.23.218.194:34241)
I20260809 18:51:53.247632 28367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:56343
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:51:53.266403 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34693
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:37405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:53.344988 28923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:53.345180 28923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:53.345201 28923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:53.346328 28923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:53.346390 28923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:53.346414 28923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:51:53.347844 28923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:37405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:53.348052 28923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:53.348253 28923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:53.352903 28932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:53.352952 28923 server_base.cc:1034] running on GCE node
W20260809 18:51:53.352923 28935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:53.352918 28931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:53.353243 28923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:53.353468 28923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:53.354610 28923 hybrid_clock.cc:648] HybridClock initialized: now 1786301513354595 us; error 29 us; skew 500 ppm
Aug 09 18:51:53 dist-test-slave-6zbq krb5kdc[28303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301513, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:53.357543 28923 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:51:53.358016 28923 webserver.cc:460] Webserver started at http://127.23.218.195:35649/ using document root <none> and password file <none>
I20260809 18:51:53.358282 28923 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:53.358413 28923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:53.358560 28923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:53.359548 28923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "3e6df11edeb44d0aa17ae5d0d3d9fc10"
format_stamp: "Formatted at 2026-08-09 18:51:53 on dist-test-slave-6zbq"
server_key: "d96975ad79abf2e62e8fbfbc39d831bc"
server_key_iv: "d457b08e4f404b1384033cf7330a432c"
server_key_version: "encryptionkey@0"
I20260809 18:51:53.359874 28923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "3e6df11edeb44d0aa17ae5d0d3d9fc10"
format_stamp: "Formatted at 2026-08-09 18:51:53 on dist-test-slave-6zbq"
server_key: "d96975ad79abf2e62e8fbfbc39d831bc"
server_key_iv: "d457b08e4f404b1384033cf7330a432c"
server_key_version: "encryptionkey@0"
I20260809 18:51:53.361171 28923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:51:53.361865 28945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.362003 28923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:53.362088 28923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "3e6df11edeb44d0aa17ae5d0d3d9fc10"
format_stamp: "Formatted at 2026-08-09 18:51:53 on dist-test-slave-6zbq"
server_key: "d96975ad79abf2e62e8fbfbc39d831bc"
server_key_iv: "d457b08e4f404b1384033cf7330a432c"
server_key_version: "encryptionkey@0"
I20260809 18:51:53.362154 28923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:53.382963 28923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:53.384541 28923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:53.384663 28923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:53.384858 28923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:53.385133 28923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:53.385164 28923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.385191 28923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:53.385208 28923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:53.390799 28923 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:41627
I20260809 18:51:53.390854 29131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:41627 every 8 connection(s)
I20260809 18:51:53.391120 28923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
Aug 09 18:51:53 dist-test-slave-6zbq krb5kdc[28303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301513, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:53.398941 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:53.391785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:44763 (local address 127.23.218.254:34693)
0809 18:51:53.391826 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:53.391828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:53.391994 (+ 166us) server_negotiation.cc:407] Connection header received
0809 18:51:53.392446 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:53.392447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:53.392457 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:53.392503 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:53.393345 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.393826 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.394739 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.394925 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.394968 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:53.396778 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:53.396780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:53.396780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:53.396787 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:53.397855 (+ 1068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:53.398202 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:53.398203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:53.398203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:53.398255 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:53.398469 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:53.398469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:53.398470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:53.398563 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:53.398611 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:53.398823 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 18:51:53.398881 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:51:53.399122 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:53.391748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34693 (local address 127.23.218.195:44763)
0809 18:51:53.391896 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:53.391929 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:53.392374 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:53.392554 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:53.392557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:53.393082 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:53.393279 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:53.393284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.393886 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.393887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:53.394670 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:53.394673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:53.394977 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:53.394977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:53.395068 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:53.395466 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:53.395474 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:53.396699 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:53.397922 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:53.397928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:53.397930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:53.398149 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:53.398297 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:53.398297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:53.398298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:53.398422 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:53.398617 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:53.398623 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:53.398730 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 18:51:53.398808 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 18:51:53.398918 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260809 18:51:53.399554 29132 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34693
I20260809 18:51:53.399633 29132 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:53.399789 29132 heartbeater.cc:511] Master 127.23.218.254:34693 requested a full tablet report, sending...
I20260809 18:51:53.400152 28367 ts_manager.cc:194] Registered new tserver with Master: 3e6df11edeb44d0aa17ae5d0d3d9fc10 (127.23.218.195:41627)
I20260809 18:51:53.400512 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 28923
I20260809 18:51:53.400514 28367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:44763
I20260809 18:51:53.400609 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:51:53.400807 24427 external_mini_cluster.cc:1428] Setting key f3435f875381d8cc04a5959613f21b96
I20260809 18:51:53.401633 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 18:51:53.413060 28433 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:51:53.412244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43850 (local address 127.23.218.254:34693)
0809 18:51:53.412289 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:53.412291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:53.412386 (+ 95us) server_negotiation.cc:407] Connection header received
0809 18:51:53.412447 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:53.412448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:53.412458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:53.412501 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:53.413004 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43850: BlockingRecv error: recv got EOF from 127.0.0.1:43850 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 18:51:53.413964 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 28469
I20260809 18:51:53.435127 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 28699
I20260809 18:51:53.443063 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 28923
I20260809 18:51:53.459479 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 28319
2026-08-09T18:51:53Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1926 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:51:53 dist-test-slave-6zbq krb5kdc[29151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:51:53 dist-test-slave-6zbq krb5kdc[29151](info): set up 2 sockets
Aug 09 18:51:53 dist-test-slave-6zbq krb5kdc[29151](info): commencing operation
krb5kdc: starting...
W20260809 18:51:54.579099 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:51:54 dist-test-slave-6zbq krb5kdc[29151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:51:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:51:54.685009 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:39497
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:34561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:39497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:54.761762 29172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:54.761945 29172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:54.761974 29172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:54.763113 29172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:51:54.763159 29172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:54.763180 29172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:54.763195 29172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:51:54.763211 29172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:51:54.764654 29172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:39497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:39497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:54.764878 29172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:54.765067 29172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:54.769481 29177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:54.769490 29183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:54.769555 29172 server_base.cc:1034] running on GCE node
W20260809 18:51:54.769505 29179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:54.769853 29172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:54.770115 29172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:54.771253 29172 hybrid_clock.cc:648] HybridClock initialized: now 1786301514771240 us; error 27 us; skew 500 ppm
Aug 09 18:51:54 dist-test-slave-6zbq krb5kdc[29151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301514, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:54.773432 29172 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:51:54.773811 29172 webserver.cc:460] Webserver started at http://127.23.218.254:37469/ using document root <none> and password file <none>
I20260809 18:51:54.773988 29172 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:54.774024 29172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:54.774111 29172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:54.775050 29172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "9de581b9c3124626bf2acc8cd55d147f"
format_stamp: "Formatted at 2026-08-09 18:51:54 on dist-test-slave-6zbq"
server_key: "818d46021193200a2b7c34a82fb55c84"
server_key_iv: "cbdbefc2be7f9d4cbc416bf44edeccfd"
server_key_version: "encryptionkey@0"
I20260809 18:51:54.775337 29172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "9de581b9c3124626bf2acc8cd55d147f"
format_stamp: "Formatted at 2026-08-09 18:51:54 on dist-test-slave-6zbq"
server_key: "818d46021193200a2b7c34a82fb55c84"
server_key_iv: "cbdbefc2be7f9d4cbc416bf44edeccfd"
server_key_version: "encryptionkey@0"
I20260809 18:51:54.776543 29172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:51:54.777133 29193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:54.777287 29172 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:54.777365 29172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "9de581b9c3124626bf2acc8cd55d147f"
format_stamp: "Formatted at 2026-08-09 18:51:54 on dist-test-slave-6zbq"
server_key: "818d46021193200a2b7c34a82fb55c84"
server_key_iv: "cbdbefc2be7f9d4cbc416bf44edeccfd"
server_key_version: "encryptionkey@0"
I20260809 18:51:54.777417 29172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:54.799273 29172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:54.801118 29172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:54.801226 29172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:54.804698 29172 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:39497
I20260809 18:51:54.804731 29289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:39497 every 8 connection(s)
I20260809 18:51:54.805207 29172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:51:54.805778 29290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:54.807821 29290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f: Bootstrap starting.
I20260809 18:51:54.808350 29290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:54.808604 29290 log.cc:824] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:54.808889 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 29172
I20260809 18:51:54.808967 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:51:54.809127 24427 external_mini_cluster.cc:1428] Setting key aba76c283bb90a2001561e82059f76ae
I20260809 18:51:54.809242 29290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f: No bootstrap required, opened a new log
I20260809 18:51:54.811810 29290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de581b9c3124626bf2acc8cd55d147f" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39497 } }
I20260809 18:51:54.811961 29290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:54.811990 29290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9de581b9c3124626bf2acc8cd55d147f, State: Initialized, Role: FOLLOWER
I20260809 18:51:54.812105 29290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de581b9c3124626bf2acc8cd55d147f" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39497 } }
I20260809 18:51:54.812170 29290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:51:54.812203 29290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:51:54.812243 29290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:54.812812 29290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de581b9c3124626bf2acc8cd55d147f" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39497 } }
I20260809 18:51:54.812899 29290 leader_election.cc:302] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9de581b9c3124626bf2acc8cd55d147f; no voters:
I20260809 18:51:54.813042 29290 leader_election.cc:288] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:51:54.813081 29300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:54.813249 29300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [term 1 LEADER]: Becoming Leader. State: Replica: 9de581b9c3124626bf2acc8cd55d147f, State: Running, Role: LEADER
I20260809 18:51:54.813283 29290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:51:54.813369 29300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de581b9c3124626bf2acc8cd55d147f" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39497 } }
I20260809 18:51:54.813685 29300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9de581b9c3124626bf2acc8cd55d147f. Latest consensus state: current_term: 1 leader_uuid: "9de581b9c3124626bf2acc8cd55d147f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de581b9c3124626bf2acc8cd55d147f" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39497 } } }
I20260809 18:51:54.813755 29300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:54.814004 29302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9de581b9c3124626bf2acc8cd55d147f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9de581b9c3124626bf2acc8cd55d147f" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39497 } } }
I20260809 18:51:54.814060 29302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:54.814540 29308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:51:54.815018 29308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:51:54.816452 29308 catalog_manager.cc:1347] Generated new cluster ID: f4c61d666e5249c59e1c45ffac60af50
Aug 09 18:51:54 dist-test-slave-6zbq krb5kdc[29151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:54.816502 29308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:51:54.819830 29298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:54.809550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.23.218.254:39497)
0809 18:51:54.809723 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:54.809732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:54.809764 (+ 32us) server_negotiation.cc:407] Connection header received
0809 18:51:54.810513 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:54.810522 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:54.810582 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:54.810720 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:54.811270 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:54.812534 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:54.813405 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:54.814236 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:54.814363 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:54.816918 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:54.816922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:54.816922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:54.816934 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:54.818380 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:54.818734 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:54.818737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:54.818737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:54.818840 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:54.819004 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:54.819005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:54.819005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:54.819291 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:54.819426 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:54.819520 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 18:51:54.819651 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260809 18:51:54.820647 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:51:54.827484 29308 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:51:54.827943 29308 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:51:54.833309 29308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9de581b9c3124626bf2acc8cd55d147f: Generated new TSK 0
I20260809 18:51:54.833448 29308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:51:54.836231 29298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:54.832857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.23.218.254:39497)
0809 18:51:54.832903 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:54.832905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:54.833030 (+ 125us) server_negotiation.cc:407] Connection header received
0809 18:51:54.833092 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:54.833093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:54.833102 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:54.833146 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:54.833554 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:54.834042 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:54.835092 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:54.835287 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:54.835334 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:54.835882 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:54.835884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:54.835884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:51:54.836032 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:54.836045 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:51:54.836051 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:54.836093 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:54.836166 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 18:51:54.836186 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:51:54.838127 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 29172
2026-08-09T18:51:54Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1381 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:51:54 dist-test-slave-6zbq krb5kdc[29346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:51:54 dist-test-slave-6zbq krb5kdc[29346](info): set up 2 sockets
Aug 09 18:51:54 dist-test-slave-6zbq krb5kdc[29346](info): commencing operation
krb5kdc: starting...
W20260809 18:51:55.961879 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:51:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:51:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:51:56.064412 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:44879
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:44879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:56.141510 29368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:56.141690 29368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:56.141709 29368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:56.142818 29368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:51:56.142856 29368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:56.142869 29368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:56.142892 29368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:51:56.142915 29368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:51:56.144347 29368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:44879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:44879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:56.144555 29368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:56.144743 29368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:56.149066 29375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:56.149173 29368 server_base.cc:1034] running on GCE node
W20260809 18:51:56.149073 29377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:56.149073 29374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:56.149499 29368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:56.149737 29368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:56.150863 29368 hybrid_clock.cc:648] HybridClock initialized: now 1786301516150848 us; error 31 us; skew 500 ppm
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:56.153031 29368 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:51:56.153412 29368 webserver.cc:460] Webserver started at http://127.23.218.254:44751/ using document root <none> and password file <none>
I20260809 18:51:56.153582 29368 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:56.153618 29368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:56.153707 29368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:56.154650 29368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "2cfb453c15b54be797b72ad428252c9e"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "3b3323b7d79ffea955661f8f314eb3d6"
server_key_iv: "b3736197ac9448210a744d8f4682c6c7"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.154928 29368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "2cfb453c15b54be797b72ad428252c9e"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "3b3323b7d79ffea955661f8f314eb3d6"
server_key_iv: "b3736197ac9448210a744d8f4682c6c7"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.156015 29368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:56.156642 29385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.156803 29368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:56.156888 29368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "2cfb453c15b54be797b72ad428252c9e"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "3b3323b7d79ffea955661f8f314eb3d6"
server_key_iv: "b3736197ac9448210a744d8f4682c6c7"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.156945 29368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:56.175866 29368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:56.177389 29368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:56.177520 29368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:56.181137 29368 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:44879
I20260809 18:51:56.181180 29463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:44879 every 8 connection(s)
I20260809 18:51:56.181464 29368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:51:56.182375 29465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:56.184582 29465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e: Bootstrap starting.
I20260809 18:51:56.185117 29465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:56.185369 29465 log.cc:824] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:56.185999 29465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e: No bootstrap required, opened a new log
I20260809 18:51:56.187508 29465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfb453c15b54be797b72ad428252c9e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44879 } }
I20260809 18:51:56.187638 29465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:56.187671 29465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cfb453c15b54be797b72ad428252c9e, State: Initialized, Role: FOLLOWER
I20260809 18:51:56.187812 29465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfb453c15b54be797b72ad428252c9e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44879 } }
I20260809 18:51:56.187870 29465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:51:56.187901 29465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:51:56.187947 29465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:56.188301 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 29368
I20260809 18:51:56.188414 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:51:56.188606 24427 external_mini_cluster.cc:1428] Setting key 1119099dfdb5d4837f4c35a51b6499fc
I20260809 18:51:56.188894 29465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfb453c15b54be797b72ad428252c9e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44879 } }
I20260809 18:51:56.189030 29465 leader_election.cc:302] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cfb453c15b54be797b72ad428252c9e; no voters:
I20260809 18:51:56.189175 29465 leader_election.cc:288] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:51:56.189213 29480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:56.189380 29480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [term 1 LEADER]: Becoming Leader. State: Replica: 2cfb453c15b54be797b72ad428252c9e, State: Running, Role: LEADER
I20260809 18:51:56.189451 29465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:51:56.189503 29480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfb453c15b54be797b72ad428252c9e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44879 } }
I20260809 18:51:56.189823 29480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cfb453c15b54be797b72ad428252c9e. Latest consensus state: current_term: 1 leader_uuid: "2cfb453c15b54be797b72ad428252c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfb453c15b54be797b72ad428252c9e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44879 } } }
I20260809 18:51:56.189890 29480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:56.190685 29486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cfb453c15b54be797b72ad428252c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfb453c15b54be797b72ad428252c9e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44879 } } }
I20260809 18:51:56.190762 29486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:56.190893 29492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:51:56.191551 29492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:51:56.193090 29492 catalog_manager.cc:1347] Generated new cluster ID: 2faf02d845d147a282d62d312ed0761c
I20260809 18:51:56.193135 29492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:56.199254 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:56.189035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.23.218.254:44879)
0809 18:51:56.189224 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:56.189243 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:56.189320 (+ 77us) server_negotiation.cc:407] Connection header received
0809 18:51:56.189460 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:56.189468 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:56.189547 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:56.189692 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:56.190463 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.191713 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.192607 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.193164 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.193276 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:56.195892 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:56.195896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:56.195897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:56.195909 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:56.197689 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.198070 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.198072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.198073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.198179 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.198401 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.198401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.198402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.198686 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:56.198825 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:56.198935 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 18:51:56.199059 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260809 18:51:56.208475 29492 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:51:56.209144 29492 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:51:56.212540 29492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cfb453c15b54be797b72ad428252c9e: Generated new TSK 0
I20260809 18:51:56.212685 29492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:51:56.236181 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:44879
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:56.314527 29516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:56.314720 29516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:56.314742 29516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:56.315868 29516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:56.315908 29516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:56.316007 29516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:51:56.317674 29516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:44879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:56.317894 29516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:56.318091 29516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:56.322783 29532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:56.322757 29529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:56.322865 29516 server_base.cc:1034] running on GCE node
W20260809 18:51:56.322754 29530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:56.323185 29516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:56.323416 29516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:56.324543 29516 hybrid_clock.cc:648] HybridClock initialized: now 1786301516324536 us; error 26 us; skew 500 ppm
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:56.326846 29516 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:51:56.327222 29516 webserver.cc:460] Webserver started at http://127.23.218.193:46583/ using document root <none> and password file <none>
I20260809 18:51:56.327402 29516 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:56.327446 29516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:56.327564 29516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:56.328503 29516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "a7131e30147a49a9bcc1d3ecc073b337"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "516e6e1fa0d341d58d6bf7e19730ff67"
server_key_iv: "10939dc479dc1c825f316bda4b94d57e"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.328819 29516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "a7131e30147a49a9bcc1d3ecc073b337"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "516e6e1fa0d341d58d6bf7e19730ff67"
server_key_iv: "10939dc479dc1c825f316bda4b94d57e"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.329955 29516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:51:56.330626 29543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.330770 29516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:56.330852 29516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "a7131e30147a49a9bcc1d3ecc073b337"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "516e6e1fa0d341d58d6bf7e19730ff67"
server_key_iv: "10939dc479dc1c825f316bda4b94d57e"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.330906 29516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:56.340420 29516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:56.341928 29516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:56.342042 29516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:56.342223 29516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:56.342515 29516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:56.342549 29516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.342573 29516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:56.342605 29516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.347946 29516 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:46219
I20260809 18:51:56.348001 29726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:46219 every 8 connection(s)
I20260809 18:51:56.348246 29516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:51:56.350256 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 29516
I20260809 18:51:56.350380 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:51:56.350565 24427 external_mini_cluster.cc:1428] Setting key 7b4444358af96bffa741ddcbbd1ad54d
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:56.357157 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:56.348828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:34331 (local address 127.23.218.254:44879)
0809 18:51:56.348869 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:56.348870 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:56.349058 (+ 188us) server_negotiation.cc:407] Connection header received
0809 18:51:56.349460 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:56.349461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:56.349470 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:56.349517 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:56.350498 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.351074 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.352333 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.352519 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.352573 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:56.354524 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:56.354525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:56.354525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:56.354533 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:56.355912 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.356293 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.356294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.356295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.356347 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.356561 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.356562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.356562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.356674 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:56.356729 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:56.356978 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 18:51:56.357060 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:51:56.357280 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:56.348806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:44879 (local address 127.23.218.193:34331)
0809 18:51:56.348949 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:56.348985 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:56.349390 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:56.349567 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:56.349569 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:56.350078 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:56.350376 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:56.350382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.351156 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.351157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:56.352260 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:56.352264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.352584 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.352585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:56.352723 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:56.353152 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:56.353162 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:56.354445 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:56.355992 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:56.356004 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:56.356006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:56.356225 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:56.356403 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:56.356404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:56.356405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:56.356504 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:56.356732 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:56.356739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:56.356863 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 18:51:56.356947 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 18:51:56.357090 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260809 18:51:56.357667 29727 heartbeater.cc:347] Connected to a master server at 127.23.218.254:44879
I20260809 18:51:56.357748 29727 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:56.357903 29727 heartbeater.cc:511] Master 127.23.218.254:44879 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
I20260809 18:51:56.358417 29411 ts_manager.cc:194] Registered new tserver with Master: a7131e30147a49a9bcc1d3ecc073b337 (127.23.218.193:46219)
I20260809 18:51:56.358994 29411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:34331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:51:56.374261 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:44879
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:56.451911 29740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:56.452085 29740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:56.452114 29740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:56.453145 29740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:56.453176 29740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:56.453199 29740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:51:56.454581 29740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:44879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:56.454754 29740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:56.454910 29740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:56.459220 29746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:56.459257 29748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:56.459365 29740 server_base.cc:1034] running on GCE node
W20260809 18:51:56.459372 29751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:56.459628 29740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:56.459854 29740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:56.460991 29740 hybrid_clock.cc:648] HybridClock initialized: now 1786301516460974 us; error 29 us; skew 500 ppm
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:56.463181 29740 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:51:56.463572 29740 webserver.cc:460] Webserver started at http://127.23.218.194:34629/ using document root <none> and password file <none>
I20260809 18:51:56.463775 29740 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:56.463821 29740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:56.463936 29740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:56.464852 29740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "1267425337ed4ee694c3ccc462db08e1"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "fa15958212a09e55fc1b3242f4d6ff7e"
server_key_iv: "424359e2bed91b54c7af799a2fed5e6a"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.465154 29740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "1267425337ed4ee694c3ccc462db08e1"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "fa15958212a09e55fc1b3242f4d6ff7e"
server_key_iv: "424359e2bed91b54c7af799a2fed5e6a"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.466259 29740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:51:56.466918 29764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.467054 29740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.467132 29740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "1267425337ed4ee694c3ccc462db08e1"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "fa15958212a09e55fc1b3242f4d6ff7e"
server_key_iv: "424359e2bed91b54c7af799a2fed5e6a"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.467191 29740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:56.481562 29740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:56.483085 29740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:56.483203 29740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:56.483378 29740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:56.483633 29740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:56.483664 29740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.483690 29740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:56.483707 29740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.489301 29740 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:34655
I20260809 18:51:56.489338 29947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:34655 every 8 connection(s)
I20260809 18:51:56.489588 29740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:56.497237 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:56.490220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:53407 (local address 127.23.218.254:44879)
0809 18:51:56.490272 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:56.490274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:56.490465 (+ 191us) server_negotiation.cc:407] Connection header received
0809 18:51:56.490871 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:56.490872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:56.490881 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:56.490932 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:56.491763 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.492245 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.493107 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.493286 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.493331 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:56.495137 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:56.495139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:56.495139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:56.495146 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:56.496251 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.496572 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.496572 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.496573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.496627 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.496801 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.496801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.496802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.496893 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:56.496936 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:56.497114 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 18:51:56.497175 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 18:51:56.497390 29950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:56.490196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:44879 (local address 127.23.218.194:53407)
0809 18:51:56.490355 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:56.490393 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:56.490802 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:56.490976 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:56.490979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:56.491522 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:56.491696 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:56.491700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.492307 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.492307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:56.493045 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:56.493047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.493338 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.493339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:56.493435 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:56.493799 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:56.493807 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:56.495047 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:56.496315 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:56.496321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:56.496323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:56.496518 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:56.496671 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:56.496671 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:56.496671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:56.496759 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:56.496933 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:56.496935 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:56.497029 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 18:51:56.497098 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 18:51:56.497210 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260809 18:51:56.497836 29948 heartbeater.cc:347] Connected to a master server at 127.23.218.254:44879
I20260809 18:51:56.497916 29948 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:56.498044 29948 heartbeater.cc:511] Master 127.23.218.254:44879 requested a full tablet report, sending...
I20260809 18:51:56.498329 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 29740
I20260809 18:51:56.498445 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:51:56.498430 29411 ts_manager.cc:194] Registered new tserver with Master: 1267425337ed4ee694c3ccc462db08e1 (127.23.218.194:34655)
I20260809 18:51:56.498602 24427 external_mini_cluster.cc:1428] Setting key d03fbfa8388ab47fd6311868defcd554
I20260809 18:51:56.498844 29411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:53407
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:51:56.521201 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:44879
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:34989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:56.600039 29954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:56.600219 29954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:56.600239 29954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:56.601295 29954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:56.601327 29954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:56.601351 29954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:51:56.602756 29954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:44879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:56.602924 29954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:56.603092 29954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:56.608670 29962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:56.608669 29966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:56.608654 29964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:56.609005 29954 server_base.cc:1034] running on GCE node
I20260809 18:51:56.609191 29954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:56.609414 29954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:56.610551 29954 hybrid_clock.cc:648] HybridClock initialized: now 1786301516610535 us; error 31 us; skew 500 ppm
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:56.613567 29954 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:51:56.613948 29954 webserver.cc:460] Webserver started at http://127.23.218.195:34847/ using document root <none> and password file <none>
I20260809 18:51:56.614154 29954 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:56.614200 29954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:56.614318 29954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:56.615258 29954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "57a368a2eefe4c4ab05e1731647cb84c"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "2c26e1f4d91aa05009fc5173484b436c"
server_key_iv: "2ba3eb53af149bb58f0fdb97019e6627"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.615562 29954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "57a368a2eefe4c4ab05e1731647cb84c"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "2c26e1f4d91aa05009fc5173484b436c"
server_key_iv: "2ba3eb53af149bb58f0fdb97019e6627"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.616706 29954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:51:56.617311 29975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.617456 29954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:51:56.617533 29954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "57a368a2eefe4c4ab05e1731647cb84c"
format_stamp: "Formatted at 2026-08-09 18:51:56 on dist-test-slave-6zbq"
server_key: "2c26e1f4d91aa05009fc5173484b436c"
server_key_iv: "2ba3eb53af149bb58f0fdb97019e6627"
server_key_version: "encryptionkey@0"
I20260809 18:51:56.617617 29954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:56.640574 29954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:56.642274 29954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:56.642421 29954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:56.642608 29954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:56.642874 29954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:56.642907 29954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.642933 29954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:56.642951 29954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:56.648211 29954 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:40073
I20260809 18:51:56.648283 30187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:40073 every 8 connection(s)
I20260809 18:51:56.648512 29954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:56.655421 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 29954
I20260809 18:51:56.655503 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:51:56.655661 24427 external_mini_cluster.cc:1428] Setting key 060ccbdef3308a7a23d67b5962616946
I20260809 18:51:56.656364 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:56.649125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:60315 (local address 127.23.218.254:44879)
0809 18:51:56.649182 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:56.649184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:56.649358 (+ 174us) server_negotiation.cc:407] Connection header received
0809 18:51:56.649783 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:56.649784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:56.649794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:56.649843 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:56.650651 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.651124 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.652001 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.652189 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.652244 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:56.653962 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:56.653963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:56.653963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:56.653970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:56.655038 (+ 1068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.655389 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.655390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.655390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.655451 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.655692 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.655693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.655693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.655823 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:56.655880 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:56.656188 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 18:51:56.656281 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 18:51:56.656534 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:56.649092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:44879 (local address 127.23.218.195:60315)
0809 18:51:56.649240 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:56.649276 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:56.649712 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:56.649892 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:56.649895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:56.650413 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:56.650586 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:56.650591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.651182 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.651183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:56.651928 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:56.651930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.652244 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.652245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:56.652340 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:56.652690 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:56.652698 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:56.653886 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:56.655106 (+ 1220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:56.655112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:56.655114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:56.655311 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:56.655507 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:56.655508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:56.655508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:56.655626 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:56.655881 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:56.655888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:56.656028 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 18:51:56.656157 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 18:51:56.656309 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260809 18:51:56.657083 30188 heartbeater.cc:347] Connected to a master server at 127.23.218.254:44879
I20260809 18:51:56.657157 30188 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:56.657285 30188 heartbeater.cc:511] Master 127.23.218.254:44879 requested a full tablet report, sending...
I20260809 18:51:56.657581 29409 ts_manager.cc:194] Registered new tserver with Master: 57a368a2eefe4c4ab05e1731647cb84c (127.23.218.195:40073)
I20260809 18:51:56.657907 29409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:60315
I20260809 18:51:56.667121 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[29346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301516, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:56.687968 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:56.681965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49806 (local address 127.23.218.254:44879)
0809 18:51:56.682021 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:56.682024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:56.682109 (+ 85us) server_negotiation.cc:407] Connection header received
0809 18:51:56.682175 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:56.682176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:56.682185 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:56.682230 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:56.682765 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.683223 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.683895 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:56.684075 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:56.684127 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:56.686051 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:56.686052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:56.686053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:56.686059 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:56.687205 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.687454 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.687455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.687455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.687496 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:56.687608 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:56.687609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:56.687609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:56.687722 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:56.687795 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:56.687861 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 18:51:56.687912 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 18:51:56.688153 29409 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49806
I20260809 18:51:56.688491 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 29516
I20260809 18:51:56.707171 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 29740
I20260809 18:51:56.713523 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 29954
I20260809 18:51:56.735319 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 29368
2026-08-09T18:51:56Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1895 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[30205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[30205](info): set up 2 sockets
Aug 09 18:51:56 dist-test-slave-6zbq krb5kdc[30205](info): commencing operation
krb5kdc: starting...
W20260809 18:51:57.873698 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:51:57 dist-test-slave-6zbq krb5kdc[30205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:51:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:51:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:51:57.978403 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:37759
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:37759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:58.056264 30221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:58.056433 30221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:58.056452 30221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:58.057533 30221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:51:58.057569 30221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:58.057582 30221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:58.057605 30221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:51:58.057626 30221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:51:58.059109 30221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:37759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:37759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:58.059337 30221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:58.059525 30221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:58.063926 30239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:58.063939 30237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:58.063933 30232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:58.064005 30221 server_base.cc:1034] running on GCE node
I20260809 18:51:58.064323 30221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:58.064584 30221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:58.065718 30221 hybrid_clock.cc:648] HybridClock initialized: now 1786301518065705 us; error 29 us; skew 500 ppm
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:58.072571 30221 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:51:58.072983 30221 webserver.cc:460] Webserver started at http://127.23.218.254:38219/ using document root <none> and password file <none>
I20260809 18:51:58.073191 30221 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:58.073239 30221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:58.073357 30221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:58.074301 30221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "1a44bced05a445f9924209a63e48177c"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "9c80fd6df509cc3026bcd879909bc451"
server_key_iv: "71ada484c39b63b1e47359b0199e05a0"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.074666 30221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "1a44bced05a445f9924209a63e48177c"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "9c80fd6df509cc3026bcd879909bc451"
server_key_iv: "71ada484c39b63b1e47359b0199e05a0"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.075815 30221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:58.076447 30250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.076588 30221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:58.076675 30221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "1a44bced05a445f9924209a63e48177c"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "9c80fd6df509cc3026bcd879909bc451"
server_key_iv: "71ada484c39b63b1e47359b0199e05a0"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.076740 30221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:58.107277 30221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:58.108944 30221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:58.109104 30221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:58.112807 30221 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:37759
I20260809 18:51:58.112851 30348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:37759 every 8 connection(s)
I20260809 18:51:58.113286 30221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:51:58.113900 30349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:51:58.116133 30349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c: Bootstrap starting.
I20260809 18:51:58.116701 30349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c: Neither blocks nor log segments found. Creating new log.
I20260809 18:51:58.116959 30349 log.cc:824] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c: Log is configured to *not* fsync() on all Append() calls
I20260809 18:51:58.117615 30349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c: No bootstrap required, opened a new log
I20260809 18:51:58.119190 30349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a44bced05a445f9924209a63e48177c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37759 } }
I20260809 18:51:58.119324 30349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:51:58.119357 30349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a44bced05a445f9924209a63e48177c, State: Initialized, Role: FOLLOWER
I20260809 18:51:58.119493 30349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a44bced05a445f9924209a63e48177c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37759 } }
I20260809 18:51:58.119552 30349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:51:58.119573 30349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:51:58.119617 30349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:51:58.120494 30349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a44bced05a445f9924209a63e48177c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37759 } }
I20260809 18:51:58.120584 30349 leader_election.cc:302] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a44bced05a445f9924209a63e48177c; no voters:
I20260809 18:51:58.120735 30349 leader_election.cc:288] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:51:58.120770 30353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:51:58.120925 30353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [term 1 LEADER]: Becoming Leader. State: Replica: 1a44bced05a445f9924209a63e48177c, State: Running, Role: LEADER
I20260809 18:51:58.120962 30349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:51:58.121013 30353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a44bced05a445f9924209a63e48177c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37759 } }
I20260809 18:51:58.121392 30355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a44bced05a445f9924209a63e48177c. Latest consensus state: current_term: 1 leader_uuid: "1a44bced05a445f9924209a63e48177c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a44bced05a445f9924209a63e48177c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37759 } } }
I20260809 18:51:58.121485 30355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:58.121373 30354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a44bced05a445f9924209a63e48177c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a44bced05a445f9924209a63e48177c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37759 } } }
I20260809 18:51:58.121663 30354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c [sys.catalog]: This master's current role is: LEADER
I20260809 18:51:58.121891 30364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:51:58.122426 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 30221
I20260809 18:51:58.122490 30364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:51:58.122524 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:51:58.122701 24427 external_mini_cluster.cc:1428] Setting key b6aad747df23e61a0c96f253bab1ee7b
I20260809 18:51:58.123980 30364 catalog_manager.cc:1347] Generated new cluster ID: 5a39895202754186b3aef3983b9cba4f
I20260809 18:51:58.124028 30364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:58.133569 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:58.123283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48400 (local address 127.23.218.254:37759)
0809 18:51:58.123508 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:58.123515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:58.123538 (+ 23us) server_negotiation.cc:407] Connection header received
0809 18:51:58.123648 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:58.123655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:58.123708 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:58.123851 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:58.124575 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.125946 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.127022 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.127622 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.127717 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:58.130280 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:58.130284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:58.130284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:58.130296 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:58.132097 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.132446 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.132449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.132449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.132549 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.132712 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.132713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.132713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.132989 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:58.133129 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:58.133207 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 18:51:58.133346 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":65,"threads_started":1}
I20260809 18:51:58.136513 30364 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:51:58.137157 30364 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:51:58.144407 30364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a44bced05a445f9924209a63e48177c: Generated new TSK 0
I20260809 18:51:58.144556 30364 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:51:58.173177 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37759
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:58.251330 30384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:58.251506 30384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:58.251538 30384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:58.252626 30384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:58.252669 30384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:58.252703 30384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:51:58.254140 30384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:58.254366 30384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:58.254558 30384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:58.259012 30395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:58.259004 30391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:58.259004 30393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:58.259088 30384 server_base.cc:1034] running on GCE node
I20260809 18:51:58.259382 30384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:58.259618 30384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:58.260746 30384 hybrid_clock.cc:648] HybridClock initialized: now 1786301518260731 us; error 32 us; skew 500 ppm
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:58.263010 30384 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:51:58.263381 30384 webserver.cc:460] Webserver started at http://127.23.218.193:43891/ using document root <none> and password file <none>
I20260809 18:51:58.263556 30384 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:58.263597 30384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:58.263688 30384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:58.264603 30384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "d9feacd4b92e45de9d28073604eff05d"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "186a318d1174cf5a92d502110dfa730c"
server_key_iv: "9a440b9ecd8fbbf1b439e72b55346379"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.264901 30384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "d9feacd4b92e45de9d28073604eff05d"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "186a318d1174cf5a92d502110dfa730c"
server_key_iv: "9a440b9ecd8fbbf1b439e72b55346379"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.266052 30384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:51:58.266716 30408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.266858 30384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.266942 30384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "d9feacd4b92e45de9d28073604eff05d"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "186a318d1174cf5a92d502110dfa730c"
server_key_iv: "9a440b9ecd8fbbf1b439e72b55346379"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.266993 30384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:58.286293 30384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:58.287921 30384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:58.288021 30384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:58.288189 30384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:58.288425 30384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:58.288455 30384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.288473 30384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:58.288489 30384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.293638 30384 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:42889
I20260809 18:51:58.293675 30604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:42889 every 8 connection(s)
I20260809 18:51:58.293944 30384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:51:58.297725 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 30384
I20260809 18:51:58.297837 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:51:58.298017 24427 external_mini_cluster.cc:1428] Setting key 32401ba73b5ee570b8ff283b27d05926
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:58.302407 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:58.294609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:55727 (local address 127.23.218.254:37759)
0809 18:51:58.294649 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:58.294651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:58.294840 (+ 189us) server_negotiation.cc:407] Connection header received
0809 18:51:58.295667 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:58.295668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:58.295681 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:58.295723 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:58.296253 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.296733 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.297606 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.297897 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.297967 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:58.299942 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:58.299944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:58.299944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:58.299954 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:58.301176 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.301526 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.301527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.301527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.301581 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.301791 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.301792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.301793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.301899 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:58.301953 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:58.302196 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 18:51:58.302285 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:51:58.302526 30609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:58.294591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:37759 (local address 127.23.218.193:55727)
0809 18:51:58.294727 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:58.294761 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:58.295579 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:58.295780 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:58.295783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:58.296006 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:58.296184 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:58.296189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.296794 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.296795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:58.297539 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:58.297540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.297973 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.297974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:58.298087 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:58.298548 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:58.298557 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:58.299855 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:58.301249 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:58.301256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:58.301258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:58.301462 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:58.301633 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:58.301634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:58.301634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:58.301737 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:58.301953 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:58.301959 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:58.302082 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 18:51:58.302165 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 18:51:58.302306 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260809 18:51:58.303001 30605 heartbeater.cc:347] Connected to a master server at 127.23.218.254:37759
I20260809 18:51:58.303089 30605 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:58.303247 30605 heartbeater.cc:511] Master 127.23.218.254:37759 requested a full tablet report, sending...
I20260809 18:51:58.303783 30290 ts_manager.cc:194] Registered new tserver with Master: d9feacd4b92e45de9d28073604eff05d (127.23.218.193:42889)
I20260809 18:51:58.304679 30290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:55727
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:51:58.320919 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37759
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:58.399718 30614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:58.399890 30614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:58.399911 30614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:58.400982 30614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:58.401016 30614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:58.401050 30614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:51:58.402488 30614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:58.402664 30614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:58.402817 30614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:58.407245 30633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:58.407281 30625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:58.407371 30639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:58.407569 30614 server_base.cc:1034] running on GCE node
I20260809 18:51:58.407778 30614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:58.407989 30614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:58.409132 30614 hybrid_clock.cc:648] HybridClock initialized: now 1786301518409115 us; error 27 us; skew 500 ppm
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:58.411446 30614 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:51:58.411834 30614 webserver.cc:460] Webserver started at http://127.23.218.194:38153/ using document root <none> and password file <none>
I20260809 18:51:58.412010 30614 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:58.412055 30614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:58.412181 30614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:58.413092 30614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "140cd1779d6242aba75488d4ef22324e"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "db663bcdbfb21586d92dd96e00f97765"
server_key_iv: "4855f72325fb472354eee5868513648d"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.413404 30614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "140cd1779d6242aba75488d4ef22324e"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "db663bcdbfb21586d92dd96e00f97765"
server_key_iv: "4855f72325fb472354eee5868513648d"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.414572 30614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:51:58.415220 30647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.415359 30614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.415438 30614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "140cd1779d6242aba75488d4ef22324e"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "db663bcdbfb21586d92dd96e00f97765"
server_key_iv: "4855f72325fb472354eee5868513648d"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.415493 30614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:58.429666 30614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:58.431574 30614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:58.431696 30614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:58.431878 30614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:58.432122 30614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:58.432160 30614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.432186 30614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:58.432210 30614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.437448 30614 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:41545
I20260809 18:51:58.437500 30837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:41545 every 8 connection(s)
I20260809 18:51:58.437741 30614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:58.445103 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 30614
I20260809 18:51:58.445186 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:51:58.445355 24427 external_mini_cluster.cc:1428] Setting key f14c11e795983facf307f3442ad35d4f
I20260809 18:51:58.445854 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:58.438363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:36493 (local address 127.23.218.254:37759)
0809 18:51:58.438410 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:58.438412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:58.438580 (+ 168us) server_negotiation.cc:407] Connection header received
0809 18:51:58.439402 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:58.439403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:58.439416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:58.439466 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:58.440006 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.440496 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.441429 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.441620 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.441676 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:58.443506 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:58.443507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:58.443507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:58.443515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:58.444661 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.444998 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.444999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.445000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.445049 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.445259 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.445260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.445260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.445363 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:58.445418 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:58.445688 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 18:51:58.445774 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:51:58.445928 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:58.438306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:37759 (local address 127.23.218.194:36493)
0809 18:51:58.438477 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:58.438509 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:58.439306 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:58.439527 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:58.439531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:58.439759 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:58.439922 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:58.439926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.440574 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.440575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:58.441349 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:58.441351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.441689 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.441690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:58.441804 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:58.442190 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:58.442197 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:58.443423 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:58.444733 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:58.444740 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:58.444742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:58.444936 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:58.445101 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:58.445101 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:58.445101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:58.445208 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:58.445416 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:58.445423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:58.445543 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 18:51:58.445651 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 18:51:58.445774 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260809 18:51:58.446326 30839 heartbeater.cc:347] Connected to a master server at 127.23.218.254:37759
I20260809 18:51:58.446440 30839 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:58.446591 30839 heartbeater.cc:511] Master 127.23.218.254:37759 requested a full tablet report, sending...
I20260809 18:51:58.446942 30290 ts_manager.cc:194] Registered new tserver with Master: 140cd1779d6242aba75488d4ef22324e (127.23.218.194:41545)
I20260809 18:51:58.447284 30290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:36493
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:51:58.466791 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37759
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:51:58.544690 30846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:51:58.544871 30846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:51:58.544903 30846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:51:58.546241 30846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:51:58.546310 30846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:51:58.546379 30846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:51:58.547878 30846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:51:58.548084 30846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:51:58.548280 30846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:51:58.552855 30856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:58.552852 30855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:51:58.552886 30860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:51:58.552891 30846 server_base.cc:1034] running on GCE node
I20260809 18:51:58.553222 30846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:51:58.553436 30846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:51:58.554565 30846 hybrid_clock.cc:648] HybridClock initialized: now 1786301518554550 us; error 27 us; skew 500 ppm
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:51:58.556742 30846 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:51:58.557111 30846 webserver.cc:460] Webserver started at http://127.23.218.195:43823/ using document root <none> and password file <none>
I20260809 18:51:58.557298 30846 fs_manager.cc:359] Metadata directory not provided
I20260809 18:51:58.557343 30846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:51:58.557458 30846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:51:58.558436 30846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "b035807ea99d44c6b622df75ff6c25ab"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "aee442e85d762f28890e37bb180570c4"
server_key_iv: "91c2dceeff3158ac715a37786bfd0c8b"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.558763 30846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "b035807ea99d44c6b622df75ff6c25ab"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "aee442e85d762f28890e37bb180570c4"
server_key_iv: "91c2dceeff3158ac715a37786bfd0c8b"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.559924 30846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:51:58.560564 30874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.560702 30846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:51:58.560779 30846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "b035807ea99d44c6b622df75ff6c25ab"
format_stamp: "Formatted at 2026-08-09 18:51:58 on dist-test-slave-6zbq"
server_key: "aee442e85d762f28890e37bb180570c4"
server_key_iv: "91c2dceeff3158ac715a37786bfd0c8b"
server_key_version: "encryptionkey@0"
I20260809 18:51:58.560838 30846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:51:58.569284 30846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:51:58.571174 30846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:51:58.571300 30846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:51:58.571496 30846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:51:58.571750 30846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:51:58.571782 30846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.571807 30846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:51:58.571825 30846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:51:58.577224 30846 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:35765
I20260809 18:51:58.577279 31054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:35765 every 8 connection(s)
I20260809 18:51:58.577507 30846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:51:58.580538 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 30846
I20260809 18:51:58.580631 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:51:58.580793 24427 external_mini_cluster.cc:1428] Setting key 84ce68c2775c0502a3241d91322f5aee
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:58.585602 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:58.578091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:47661 (local address 127.23.218.254:37759)
0809 18:51:58.578145 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:58.578147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:58.578322 (+ 175us) server_negotiation.cc:407] Connection header received
0809 18:51:58.579221 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:58.579222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:58.579231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:58.579290 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:58.579784 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.580302 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.581371 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.581611 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.581672 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:58.583473 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:58.583473 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:58.583474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:58.583480 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:58.584485 (+ 1005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.584801 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.584801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.584801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.584842 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.585085 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.585086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.585087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.585198 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:58.585275 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:58.585455 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 18:51:58.585513 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:51:58.585696 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:58.578067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:37759 (local address 127.23.218.195:47661)
0809 18:51:58.578223 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 18:51:58.578258 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 18:51:58.579142 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:51:58.579341 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:51:58.579344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:51:58.579547 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:51:58.579719 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:58.579723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.580362 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.580363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:58.581298 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:51:58.581302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.581688 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.581689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:51:58.581788 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:58.582178 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:51:58.582186 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:51:58.583400 (+ 1214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:51:58.584546 (+ 1146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:58.584552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:58.584553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:58.584749 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:58.584920 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:51:58.584920 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:51:58.584921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:51:58.585009 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:51:58.585267 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:51:58.585270 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:51:58.585363 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 18:51:58.585433 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 18:51:58.585535 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260809 18:51:58.586073 31055 heartbeater.cc:347] Connected to a master server at 127.23.218.254:37759
I20260809 18:51:58.586148 31055 heartbeater.cc:464] Registering TS with master...
I20260809 18:51:58.586298 31055 heartbeater.cc:511] Master 127.23.218.254:37759 requested a full tablet report, sending...
I20260809 18:51:58.586642 30290 ts_manager.cc:194] Registered new tserver with Master: b035807ea99d44c6b622df75ff6c25ab (127.23.218.195:35765)
I20260809 18:51:58.586958 30290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:47661
I20260809 18:51:58.592124 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:51:58.615484 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:58.609015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34314 (local address 127.23.218.193:42889)
0809 18:51:58.609171 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:58.609176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:58.609188 (+ 12us) server_negotiation.cc:407] Connection header received
0809 18:51:58.609213 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:58.609215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:58.609228 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:58.609288 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:58.609786 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.610593 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.611255 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.611536 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.611583 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:58.613443 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:58.613450 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:58.613450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:58.613455 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:58.614691 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.614962 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.614965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.614965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.615011 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.615124 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.615125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.615125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.615226 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:58.615311 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:58.615360 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 18:51:58.615414 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[30205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:51:58.622521 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:51:58.616570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48410 (local address 127.23.218.254:37759)
0809 18:51:58.616621 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 18:51:58.616623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:51:58.616630 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:51:58.616686 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:51:58.616687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:51:58.616696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:51:58.616739 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:51:58.617137 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.617588 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.618250 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:51:58.618485 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:51:58.618537 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:51:58.620294 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:51:58.620295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:51:58.620295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:51:58.620302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:51:58.621422 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.621770 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.621771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.621771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.621823 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:51:58.622017 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:51:58.622017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:51:58.622018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:51:58.622143 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:51:58.622220 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 18:51:58.622369 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 18:51:58.622439 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 18:51:58.622709 30290 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48410
I20260809 18:51:58.623013 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 30384
I20260809 18:51:58.639248 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 30614
I20260809 18:51:58.655902 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 30846
I20260809 18:51:58.690021 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 30221
2026-08-09T18:51:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1951 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[31076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[31076](info): set up 2 sockets
Aug 09 18:51:58 dist-test-slave-6zbq krb5kdc[31076](info): commencing operation
krb5kdc: starting...
W20260809 18:51:59.819674 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:51:59 dist-test-slave-6zbq krb5kdc[31076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:51:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:51:59.922784 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:37823
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:37823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:00.000062 31105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:00.000254 31105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:00.000275 31105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:00.001374 31105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:00.001412 31105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:00.001426 31105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:00.001449 31105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:00.001471 31105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:00.002974 31105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:37823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:37823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:00.003202 31105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:00.003386 31105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:00.007778 31122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:00.007769 31115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.007793 31105 server_base.cc:1034] running on GCE node
W20260809 18:52:00.007771 31119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.008157 31105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:00.008420 31105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:00.009552 31105 hybrid_clock.cc:648] HybridClock initialized: now 1786301520009540 us; error 23 us; skew 500 ppm
Aug 09 18:52:00 dist-test-slave-6zbq krb5kdc[31076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301520, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:00.011795 31105 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:52:00.012130 31105 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:00.012183 31105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:00.012306 31105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:00.013253 31105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "862b1af95c494dd5886f16962573bdf8"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "31b00e4c595f0bcd63f291c048e8f077"
server_key_iv: "6965860840d20eb0378403c8e08bf316"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.013571 31105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "862b1af95c494dd5886f16962573bdf8"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "31b00e4c595f0bcd63f291c048e8f077"
server_key_iv: "6965860840d20eb0378403c8e08bf316"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.014771 31105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:52:00.015405 31144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.015549 31105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:00.015630 31105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "862b1af95c494dd5886f16962573bdf8"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "31b00e4c595f0bcd63f291c048e8f077"
server_key_iv: "6965860840d20eb0378403c8e08bf316"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.015694 31105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:00.039572 31105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:00.041494 31105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:00.041622 31105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:00.045174 31105 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:37823
I20260809 18:52:00.045197 31238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:37823 every 8 connection(s)
I20260809 18:52:00.045436 31105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:00.046365 31239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:00.046509 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 31105
I20260809 18:52:00.046607 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:00.046780 24427 external_mini_cluster.cc:1428] Setting key 1b9a2466737521e749d8bbea62c2da5d
I20260809 18:52:00.049314 31239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8: Bootstrap starting.
I20260809 18:52:00.049968 31239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:00.050256 31239 log.cc:824] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:00.050899 31239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8: No bootstrap required, opened a new log
Aug 09 18:52:00 dist-test-slave-6zbq krb5kdc[31076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:00.052863 31239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b1af95c494dd5886f16962573bdf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37823 } }
I20260809 18:52:00.052994 31239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:00.053015 31239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 862b1af95c494dd5886f16962573bdf8, State: Initialized, Role: FOLLOWER
I20260809 18:52:00.053129 31239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b1af95c494dd5886f16962573bdf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37823 } }
I20260809 18:52:00.053206 31239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:00.053228 31239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:00.053275 31239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:00.053858 31239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b1af95c494dd5886f16962573bdf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37823 } }
I20260809 18:52:00.053948 31239 leader_election.cc:302] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 862b1af95c494dd5886f16962573bdf8; no voters:
I20260809 18:52:00.054107 31239 leader_election.cc:288] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:00.054152 31244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:00.054293 31244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [term 1 LEADER]: Becoming Leader. State: Replica: 862b1af95c494dd5886f16962573bdf8, State: Running, Role: LEADER
I20260809 18:52:00.054389 31239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:00.054401 31244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b1af95c494dd5886f16962573bdf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37823 } }
I20260809 18:52:00.054867 31245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "862b1af95c494dd5886f16962573bdf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b1af95c494dd5886f16962573bdf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37823 } } }
I20260809 18:52:00.054881 31246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 862b1af95c494dd5886f16962573bdf8. Latest consensus state: current_term: 1 leader_uuid: "862b1af95c494dd5886f16962573bdf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b1af95c494dd5886f16962573bdf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37823 } } }
I20260809 18:52:00.055039 31245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:00.055363 31257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:00.055055 31246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:00.055866 31257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:00.056744 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.047221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.23.218.254:37823)
0809 18:52:00.047359 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.047366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.047389 (+ 23us) server_negotiation.cc:407] Connection header received
0809 18:52:00.047815 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.047821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.047866 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.047969 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:00.048676 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.049508 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.050524 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.050917 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.051009 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.053302 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:00.053307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:00.053307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:00.053319 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:00.055095 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.055483 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.055486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.055487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.055582 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.055801 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.055801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.055802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.056080 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:00.056217 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.056442 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.056562 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260809 18:52:00.057654 31257 catalog_manager.cc:1347] Generated new cluster ID: d8b4371dd374427d9926d18e2d4f4c68
I20260809 18:52:00.057694 31257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:00.063506 31257 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:00.063961 31257 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:00.070284 31257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8: Generated new TSK 0
I20260809 18:52:00.070452 31257 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:52:00.092903 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37823
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:00.171447 31271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:00.171625 31271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:00.171646 31271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:00.172731 31271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:00.172767 31271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:00.172791 31271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:00.174202 31271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:00.174415 31271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:00.174624 31271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:00.179042 31285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:00.179046 31291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.179061 31271 server_base.cc:1034] running on GCE node
W20260809 18:52:00.179067 31288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.179425 31271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:00.179636 31271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:00.180775 31271 hybrid_clock.cc:648] HybridClock initialized: now 1786301520180765 us; error 31 us; skew 500 ppm
Aug 09 18:52:00 dist-test-slave-6zbq krb5kdc[31076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301520, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:00.183073 31271 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:52:00.183346 31271 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:00.183387 31271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:00.183473 31271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:00.184372 31271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "1f68fb4e831747ec89aff3b778fc671f"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "c146e3aba2b234a09acfc1731363e4e8"
server_key_iv: "2e926b84214d0aa3eb7633c83a3ae46f"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.184643 31271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "1f68fb4e831747ec89aff3b778fc671f"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "c146e3aba2b234a09acfc1731363e4e8"
server_key_iv: "2e926b84214d0aa3eb7633c83a3ae46f"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.185711 31271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:00.186287 31298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.186479 31271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:00.186569 31271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "1f68fb4e831747ec89aff3b778fc671f"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "c146e3aba2b234a09acfc1731363e4e8"
server_key_iv: "2e926b84214d0aa3eb7633c83a3ae46f"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.186633 31271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:00.201917 31271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:00.203805 31271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:00.203926 31271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:00.204113 31271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:00.204370 31271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:00.204402 31271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.204427 31271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:00.204452 31271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.209692 31271 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:46477
I20260809 18:52:00.209740 31486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:46477 every 8 connection(s)
I20260809 18:52:00.209941 31271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
Aug 09 18:52:00 dist-test-slave-6zbq krb5kdc[31076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301520, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:00.216856 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 31271
I20260809 18:52:00.216950 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:52:00.217125 24427 external_mini_cluster.cc:1428] Setting key eb6cc98188981e8ab0e5eb593949cec2
I20260809 18:52:00.218036 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.210544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:55493 (local address 127.23.218.254:37823)
0809 18:52:00.210586 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.210587 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.210758 (+ 171us) server_negotiation.cc:407] Connection header received
0809 18:52:00.211190 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.211191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.211203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.211253 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:00.212251 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.212736 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.213648 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.213847 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.213901 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.215646 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:00.215648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:00.215648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:00.215656 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:00.216665 (+ 1009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.217034 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.217036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.217037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.217106 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.217360 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.217361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.217361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.217493 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:00.217550 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.217842 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.217931 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:52:00.218081 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.210504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:37823 (local address 127.23.218.193:55493)
0809 18:52:00.210652 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.210684 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:00.211101 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:00.211315 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:00.211319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:00.211539 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:00.212160 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.212166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.212815 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.212816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.213564 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.213566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.213915 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.213916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.214026 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.214400 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:00.214407 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:00.215566 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:00.216734 (+ 1168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:00.216741 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:00.216743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:00.216958 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:00.217169 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:00.217169 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:00.217170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:00.217295 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:00.217552 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:00.217557 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:00.217697 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 18:52:00.217798 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 18:52:00.217920 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260809 18:52:00.219604 31489 heartbeater.cc:347] Connected to a master server at 127.23.218.254:37823
I20260809 18:52:00.219695 31489 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:00.219830 31489 heartbeater.cc:511] Master 127.23.218.254:37823 requested a full tablet report, sending...
I20260809 18:52:00.220288 31177 ts_manager.cc:194] Registered new tserver with Master: 1f68fb4e831747ec89aff3b778fc671f (127.23.218.193:46477)
I20260809 18:52:00.220857 31177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:55493
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:52:00.238078 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37823
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:00.314168 31497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:00.314363 31497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:00.314388 31497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:00.315476 31497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:00.315512 31497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:00.315538 31497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:00.316982 31497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:00.317185 31497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:00.317368 31497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:00.321971 31505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:00.322024 31506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:00.322124 31509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.322136 31497 server_base.cc:1034] running on GCE node
I20260809 18:52:00.322376 31497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:00.322592 31497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:00.323714 31497 hybrid_clock.cc:648] HybridClock initialized: now 1786301520323706 us; error 26 us; skew 500 ppm
Aug 09 18:52:00 dist-test-slave-6zbq krb5kdc[31076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301520, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:00.325915 31497 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:52:00.326184 31497 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:00.326223 31497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:00.326308 31497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:00.327343 31497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "933536aa412d4c8a90f9a9671cf38de6"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "982bdbe346baf75024e2461521f7431d"
server_key_iv: "93f62d4e4f46c6a8b27f58522ce5b3f8"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.327616 31497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "933536aa412d4c8a90f9a9671cf38de6"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "982bdbe346baf75024e2461521f7431d"
server_key_iv: "93f62d4e4f46c6a8b27f58522ce5b3f8"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.328692 31497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:00.329284 31516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.329447 31497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:00.329527 31497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "933536aa412d4c8a90f9a9671cf38de6"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "982bdbe346baf75024e2461521f7431d"
server_key_iv: "93f62d4e4f46c6a8b27f58522ce5b3f8"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.329589 31497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:00.339629 31497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:00.341473 31497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:00.341609 31497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:00.341775 31497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:00.341997 31497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:00.342026 31497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.342049 31497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:00.342065 31497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.347180 31497 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:35555
I20260809 18:52:00.347213 31701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:35555 every 8 connection(s)
I20260809 18:52:00.347424 31497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:52:00.351788 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 31497
I20260809 18:52:00.351877 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:52:00.352032 24427 external_mini_cluster.cc:1428] Setting key b201f1c96c90dd7a0ec86c3f0bdd6937
Aug 09 18:52:00 dist-test-slave-6zbq krb5kdc[31076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301520, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:00.355495 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.348016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:39971 (local address 127.23.218.254:37823)
0809 18:52:00.348101 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.348103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.348213 (+ 110us) server_negotiation.cc:407] Connection header received
0809 18:52:00.348592 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.348593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.348602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.348649 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:00.349431 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.349910 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.350816 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.350993 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.351042 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.352824 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:00.352825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:00.352826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:00.352833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:00.354124 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.354616 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.354617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.354618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.354677 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.354940 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.354940 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.354941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.355038 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:00.355085 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.355368 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.355434 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 18:52:00.355690 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.347977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:37823 (local address 127.23.218.194:39971)
0809 18:52:00.348109 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.348139 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:00.348521 (+ 382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:00.348694 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:00.348697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:00.348916 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:00.349366 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.349371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.349976 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.349977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.350756 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.350758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.351049 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.351049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.351138 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.351486 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:00.351493 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:00.352745 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:00.354212 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:00.354220 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:00.354222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:00.354520 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:00.354743 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:00.354743 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:00.354743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:00.354886 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:00.355087 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:00.355091 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:00.355243 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 18:52:00.355355 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 18:52:00.355501 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":74,"threads_started":1}
I20260809 18:52:00.356117 31702 heartbeater.cc:347] Connected to a master server at 127.23.218.254:37823
I20260809 18:52:00.356240 31702 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:00.356379 31702 heartbeater.cc:511] Master 127.23.218.254:37823 requested a full tablet report, sending...
I20260809 18:52:00.356731 31177 ts_manager.cc:194] Registered new tserver with Master: 933536aa412d4c8a90f9a9671cf38de6 (127.23.218.194:35555)
I20260809 18:52:00.357112 31177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:39971
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:52:00.374514 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37823
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:00.453329 31714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:00.453521 31714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:00.453552 31714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:00.454720 31714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:00.454756 31714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:00.454792 31714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:00.456319 31714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:00.456543 31714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:00.456733 31714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:00.461231 31723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:00.461236 31724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.461423 31714 server_base.cc:1034] running on GCE node
W20260809 18:52:00.461266 31727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.461689 31714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:00.461895 31714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:00.463022 31714 hybrid_clock.cc:648] HybridClock initialized: now 1786301520463010 us; error 29 us; skew 500 ppm
Aug 09 18:52:00 dist-test-slave-6zbq krb5kdc[31076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301520, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:00.465309 31714 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:52:00.465595 31714 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:00.465641 31714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:00.465750 31714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:00.466769 31714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "8f942b2c1bc0418d88e146376bc4c598"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "15d60da8ef19282e8be690289b617b87"
server_key_iv: "a57d160141832f29a4cc5d9b9898767d"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.467118 31714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "8f942b2c1bc0418d88e146376bc4c598"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "15d60da8ef19282e8be690289b617b87"
server_key_iv: "a57d160141832f29a4cc5d9b9898767d"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.468264 31714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:00.468868 31737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.469038 31714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:00.469121 31714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "8f942b2c1bc0418d88e146376bc4c598"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "15d60da8ef19282e8be690289b617b87"
server_key_iv: "a57d160141832f29a4cc5d9b9898767d"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.469179 31714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:00.486275 31714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:00.487938 31714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:00.488040 31714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:00.488204 31714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:00.488443 31714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:00.488473 31714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.488493 31714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:00.488512 31714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.493983 31714 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:41257
I20260809 18:52:00.494031 31930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:41257 every 8 connection(s)
I20260809 18:52:00.494264 31714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:52:00.498591 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 31714
I20260809 18:52:00.498700 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:52:00.498884 24427 external_mini_cluster.cc:1428] Setting key 3ffc2782c5330204a1ccba02b14b51ad
Aug 09 18:52:00 dist-test-slave-6zbq krb5kdc[31076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301520, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:00.502611 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.494938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:41659 (local address 127.23.218.254:37823)
0809 18:52:00.494992 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.494994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.495185 (+ 191us) server_negotiation.cc:407] Connection header received
0809 18:52:00.495660 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.495661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.495671 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.495713 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:00.496565 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.497061 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.497958 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.498132 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.498173 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.500378 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:00.500379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:00.500379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:00.500386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:00.501615 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.501919 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.501919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.501920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.501966 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.502139 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.502139 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.502140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.502228 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:00.502272 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.502490 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.502549 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:52:00.502754 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.494894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:37823 (local address 127.23.218.195:41659)
0809 18:52:00.495065 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.495104 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:00.495557 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:00.495769 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:00.495773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:00.496014 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:00.496499 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.496504 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.497121 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.497121 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.497901 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.497903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.498182 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.498183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.498283 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.498802 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:00.498812 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:00.500291 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:00.501680 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:00.501683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:00.501684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:00.501868 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:00.502008 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:00.502008 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:00.502008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:00.502098 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:00.502274 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:00.502276 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:00.502397 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 18:52:00.502468 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 18:52:00.502577 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260809 18:52:00.503196 31931 heartbeater.cc:347] Connected to a master server at 127.23.218.254:37823
I20260809 18:52:00.503291 31931 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:00.503427 31931 heartbeater.cc:511] Master 127.23.218.254:37823 requested a full tablet report, sending...
I20260809 18:52:00.503760 31177 ts_manager.cc:194] Registered new tserver with Master: 8f942b2c1bc0418d88e146376bc4c598 (127.23.218.195:41257)
I20260809 18:52:00.504097 31177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:41659
I20260809 18:52:00.510489 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:52:00.516747 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.511701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.23.218.254:37823)
0809 18:52:00.511736 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.511737 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.511856 (+ 119us) server_negotiation.cc:407] Connection header received
0809 18:52:00.511895 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.511895 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.511903 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.511943 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:00.512419 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.512857 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.513494 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.513666 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.513709 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.514706 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:00.514707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:00.514708 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:00.514714 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:00.515896 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.516200 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.516200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.516200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.516244 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:00.516367 (+ 123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:00.516368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:00.516368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:00.516492 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:00.516561 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.516629 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.516691 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:52:00.518007 31177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:52:00.519729 31177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:00.526104 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.521548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46477 (local address 127.0.0.1:51698)
0809 18:52:00.521863 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.521902 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:00.521975 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:00.522217 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:00.522220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:00.522222 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:00.522271 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.522273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.523648 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.523649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.524953 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.524956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.525827 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.525828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.525882 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.525884 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:52:00.525940 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 18:52:00.526021 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":55,"thread_start_us":63,"threads_started":1}
I20260809 18:52:00.526104 31950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.521665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51698 (local address 127.23.218.193:46477)
0809 18:52:00.521852 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.521856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.522049 (+ 193us) server_negotiation.cc:407] Connection header received
0809 18:52:00.522097 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.522099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.522114 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.522186 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:00.522334 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.523549 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.525042 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.525748 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.525816 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.525862 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.525991 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.526029 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260809 18:52:00.526180 31955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.521882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55924 (local address 127.23.218.195:41257)
0809 18:52:00.522188 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.522191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.522200 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:00.522245 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.522247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.522259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.522382 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:00.522590 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.523605 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.525143 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.525660 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.525698 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.525732 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.526065 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.526103 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":81,"threads_started":1}
I20260809 18:52:00.526355 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.521576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41257 (local address 127.0.0.1:55924)
0809 18:52:00.522039 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.522049 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:00.522093 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:00.522465 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:00.522466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:00.522468 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:00.522516 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.522517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.523688 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.523689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.525044 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.525046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.525780 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.525781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.525867 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.525879 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 18:52:00.525940 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 18:52:00.526011 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":47,"thread_start_us":48,"threads_started":1}
I20260809 18:52:00.526854 31393 tablet_service.cc:1467] Processing CreateTablet for tablet 8bd2576327484b2a898ac41f422e02e4 (DEFAULT_TABLE table=test-table [id=00b3f02e518342ac85d19be047a9d369]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:00.526932 31827 tablet_service.cc:1467] Processing CreateTablet for tablet 8bd2576327484b2a898ac41f422e02e4 (DEFAULT_TABLE table=test-table [id=00b3f02e518342ac85d19be047a9d369]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:00.527118 31393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bd2576327484b2a898ac41f422e02e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:00.527168 31827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bd2576327484b2a898ac41f422e02e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:00.529541 31958 tablet_bootstrap.cc:492] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: Bootstrap starting.
I20260809 18:52:00.529651 31957 tablet_bootstrap.cc:492] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f: Bootstrap starting.
I20260809 18:52:00.530133 31958 tablet_bootstrap.cc:654] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:00.530325 31957 tablet_bootstrap.cc:654] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:00.530776 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.521752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:35555 (local address 127.0.0.1:58152)
0809 18:52:00.523723 (+ 1971us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.523731 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:00.523775 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:00.523922 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:00.523922 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:00.523924 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:00.523964 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.523966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.524858 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.524859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.525884 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.525887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.530552 (+ 4665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.530554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.530613 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.530618 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:52:00.530649 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 18:52:00.530689 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1959,"thread_start_us":68,"threads_started":1}
I20260809 18:52:00.530772 31958 log.cc:824] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:00.530911 31957 log.cc:824] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:00.531143 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.521861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.23.218.194:35555)
0809 18:52:00.522009 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.522013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.523793 (+ 1780us) server_negotiation.cc:407] Connection header received
0809 18:52:00.523837 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.523839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.523853 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.523919 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:00.524019 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.524772 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.525945 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.530447 (+ 4502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.530943 (+ 496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.531005 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.531031 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.531070 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260809 18:52:00.531483 31958 tablet_bootstrap.cc:492] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: No bootstrap required, opened a new log
I20260809 18:52:00.531548 31958 ts_tablet_manager.cc:1397] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 18:52:00.531641 31957 tablet_bootstrap.cc:492] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f: No bootstrap required, opened a new log
I20260809 18:52:00.531703 31957 ts_tablet_manager.cc:1397] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 18:52:00.531828 31604 tablet_service.cc:1467] Processing CreateTablet for tablet 8bd2576327484b2a898ac41f422e02e4 (DEFAULT_TABLE table=test-table [id=00b3f02e518342ac85d19be047a9d369]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:00.532126 31604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bd2576327484b2a898ac41f422e02e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:00.533210 31957 raft_consensus.cc:348] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } }
I20260809 18:52:00.533335 31957 raft_consensus.cc:374] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:00.533366 31957 raft_consensus.cc:729] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f68fb4e831747ec89aff3b778fc671f, State: Initialized, Role: FOLLOWER
I20260809 18:52:00.533475 31957 consensus_queue.cc:260] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } }
I20260809 18:52:00.533420 31958 raft_consensus.cc:348] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } }
I20260809 18:52:00.533564 31958 raft_consensus.cc:374] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:00.533607 31958 raft_consensus.cc:729] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f942b2c1bc0418d88e146376bc4c598, State: Initialized, Role: FOLLOWER
I20260809 18:52:00.533715 31958 consensus_queue.cc:260] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } }
I20260809 18:52:00.533922 31957 ts_tablet_manager.cc:1428] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 18:52:00.533975 31958 ts_tablet_manager.cc:1428] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 18:52:00.534041 31489 heartbeater.cc:503] Master 127.23.218.254:37823 was elected leader, sending a full tablet report...
I20260809 18:52:00.534142 31931 heartbeater.cc:503] Master 127.23.218.254:37823 was elected leader, sending a full tablet report...
I20260809 18:52:00.535121 31966 tablet_bootstrap.cc:492] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: Bootstrap starting.
I20260809 18:52:00.535647 31966 tablet_bootstrap.cc:654] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:00.536113 31966 log.cc:824] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:00.536707 31966 tablet_bootstrap.cc:492] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: No bootstrap required, opened a new log
I20260809 18:52:00.536764 31966 ts_tablet_manager.cc:1397] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:52:00.538075 31966 raft_consensus.cc:348] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } }
I20260809 18:52:00.538214 31966 raft_consensus.cc:374] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:00.538250 31966 raft_consensus.cc:729] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 933536aa412d4c8a90f9a9671cf38de6, State: Initialized, Role: FOLLOWER
I20260809 18:52:00.538376 31966 consensus_queue.cc:260] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } }
I20260809 18:52:00.538601 31966 ts_tablet_manager.cc:1428] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:52:00.538652 31702 heartbeater.cc:503] Master 127.23.218.254:37823 was elected leader, sending a full tablet report...
W20260809 18:52:00.597891 31703 tablet.cc:2369] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:52:00.623556 31961 raft_consensus.cc:482] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:00.623665 31961 raft_consensus.cc:504] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } }
I20260809 18:52:00.623991 31961 leader_election.cc:288] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 933536aa412d4c8a90f9a9671cf38de6 (127.23.218.194:35555), 8f942b2c1bc0418d88e146376bc4c598 (127.23.218.195:41257)
I20260809 18:52:00.626720 31955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.624214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:42371 (local address 127.23.218.195:41257)
0809 18:52:00.624260 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.624262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.624329 (+ 67us) server_negotiation.cc:407] Connection header received
0809 18:52:00.624407 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.624408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.624418 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.624466 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:00.624695 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.625142 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.626028 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.626438 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.626499 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.626508 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.626626 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.626658 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:52:00.626749 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.624133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41257 (local address 127.23.218.193:42371)
0809 18:52:00.624270 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.624282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:00.624344 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:00.624585 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:00.624586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:00.624587 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:00.624638 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.624639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.625208 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.625208 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.625951 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.625953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.626505 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.626506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.626569 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.626572 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:00.626616 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 18:52:00.626652 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 18:52:00.626749 31954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.624204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:44233 (local address 127.23.218.194:35555)
0809 18:52:00.624329 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.624331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.624464 (+ 133us) server_negotiation.cc:407] Connection header received
0809 18:52:00.624493 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.624495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.624503 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.624547 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:00.624682 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.625138 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.626039 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.626452 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.626509 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.626518 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.626654 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.626688 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 18:52:00.626775 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.624134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:35555 (local address 127.23.218.193:44233)
0809 18:52:00.624403 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.624411 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:00.624461 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:00.624557 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:00.624558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:00.624559 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:00.624619 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.624620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.625213 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.625214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.625964 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:00.625966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.626528 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.626529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:00.626594 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.626597 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:00.626647 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 18:52:00.626687 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":88,"thread_start_us":132,"threads_started":1}
I20260809 18:52:00.627050 31638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bd2576327484b2a898ac41f422e02e4" candidate_uuid: "1f68fb4e831747ec89aff3b778fc671f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933536aa412d4c8a90f9a9671cf38de6" is_pre_election: true
I20260809 18:52:00.627071 31857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bd2576327484b2a898ac41f422e02e4" candidate_uuid: "1f68fb4e831747ec89aff3b778fc671f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f942b2c1bc0418d88e146376bc4c598" is_pre_election: true
I20260809 18:52:00.627187 31638 raft_consensus.cc:2454] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f68fb4e831747ec89aff3b778fc671f in term 0.
I20260809 18:52:00.627211 31857 raft_consensus.cc:2454] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f68fb4e831747ec89aff3b778fc671f in term 0.
I20260809 18:52:00.627391 31300 leader_election.cc:302] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f68fb4e831747ec89aff3b778fc671f, 933536aa412d4c8a90f9a9671cf38de6; no voters:
I20260809 18:52:00.627521 31961 raft_consensus.cc:2781] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:00.627580 31961 raft_consensus.cc:482] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:00.627604 31961 raft_consensus.cc:3037] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:00.628262 31961 raft_consensus.cc:504] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } }
I20260809 18:52:00.628376 31961 leader_election.cc:288] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [CANDIDATE]: Term 1 election: Requested vote from peers 933536aa412d4c8a90f9a9671cf38de6 (127.23.218.194:35555), 8f942b2c1bc0418d88e146376bc4c598 (127.23.218.195:41257)
I20260809 18:52:00.628520 31638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bd2576327484b2a898ac41f422e02e4" candidate_uuid: "1f68fb4e831747ec89aff3b778fc671f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "933536aa412d4c8a90f9a9671cf38de6"
I20260809 18:52:00.628520 31857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bd2576327484b2a898ac41f422e02e4" candidate_uuid: "1f68fb4e831747ec89aff3b778fc671f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f942b2c1bc0418d88e146376bc4c598"
I20260809 18:52:00.628590 31857 raft_consensus.cc:3037] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:00.628590 31638 raft_consensus.cc:3037] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:00.629423 31638 raft_consensus.cc:2454] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f68fb4e831747ec89aff3b778fc671f in term 1.
I20260809 18:52:00.629423 31857 raft_consensus.cc:2454] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f68fb4e831747ec89aff3b778fc671f in term 1.
I20260809 18:52:00.629582 31301 leader_election.cc:302] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f68fb4e831747ec89aff3b778fc671f, 8f942b2c1bc0418d88e146376bc4c598; no voters:
I20260809 18:52:00.629669 31961 raft_consensus.cc:2781] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:00.629792 31961 raft_consensus.cc:686] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [term 1 LEADER]: Becoming Leader. State: Replica: 1f68fb4e831747ec89aff3b778fc671f, State: Running, Role: LEADER
I20260809 18:52:00.629873 31961 consensus_queue.cc:237] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } }
I20260809 18:52:00.630611 31177 catalog_manager.cc:5495] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f68fb4e831747ec89aff3b778fc671f (127.23.218.193). New cstate: current_term: 1 leader_uuid: "1f68fb4e831747ec89aff3b778fc671f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f68fb4e831747ec89aff3b778fc671f" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 } health_report { overall_health: UNKNOWN } } }
I20260809 18:52:00.639165 31950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.636933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.23.218.193:46477)
0809 18:52:00.636980 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.636981 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.636991 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:52:00.637062 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.637063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.637080 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.637154 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:00.637287 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.637715 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.638482 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:00.638662 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:00.638713 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:00.638860 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:00.638949 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:00.639017 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.639052 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.639069 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 18:52:00.640591 31638 raft_consensus.cc:1261] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [term 1 FOLLOWER]: Refusing update from remote peer 1f68fb4e831747ec89aff3b778fc671f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:00.640602 31857 raft_consensus.cc:1261] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [term 1 FOLLOWER]: Refusing update from remote peer 1f68fb4e831747ec89aff3b778fc671f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:00.640893 31961 consensus_queue.cc:1035] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [LEADER]: Connected to new peer: Peer: permanent_uuid: "933536aa412d4c8a90f9a9671cf38de6" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:00.641009 31973 consensus_queue.cc:1035] T 8bd2576327484b2a898ac41f422e02e4 P 1f68fb4e831747ec89aff3b778fc671f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f942b2c1bc0418d88e146376bc4c598" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:00.643714 31982 mvcc.cc:205] Tried to move back new op lower bound from 7316691028541825024 to 7316691028500471808. Current Snapshot: MvccSnapshot[applied={T|T < 7316691028541825024}]
I20260809 18:52:00.645391 31176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:00.645478 31176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:00.646176 31176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 862b1af95c494dd5886f16962573bdf8: Sending DeleteTablet for 3 replicas of tablet 8bd2576327484b2a898ac41f422e02e4
I20260809 18:52:00.646510 31827 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bd2576327484b2a898ac41f422e02e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:00 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:55924
I20260809 18:52:00.646529 31604 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bd2576327484b2a898ac41f422e02e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:00 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:58152
I20260809 18:52:00.646538 31393 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bd2576327484b2a898ac41f422e02e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:00 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:51698
I20260809 18:52:00.646752 31990 tablet_replica.cc:330] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: stopping tablet replica
I20260809 18:52:00.646852 31990 raft_consensus.cc:2229] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:00.646919 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 31271
I20260809 18:52:00.646950 31990 raft_consensus.cc:2258] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:00.660717 31992 tablet_replica.cc:330] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: stopping tablet replica
I20260809 18:52:00.660830 31992 raft_consensus.cc:2229] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:00.660944 31992 raft_consensus.cc:2258] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:00.661028 31990 ts_tablet_manager.cc:1905] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:00.661330 31992 ts_tablet_manager.cc:1905] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:00.662569 31992 ts_tablet_manager.cc:1918] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:00.662578 31990 ts_tablet_manager.cc:1918] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:00.662633 31992 log.cc:1196] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/8bd2576327484b2a898ac41f422e02e4
I20260809 18:52:00.662639 31990 log.cc:1196] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/8bd2576327484b2a898ac41f422e02e4
I20260809 18:52:00.662869 31992 ts_tablet_manager.cc:1939] T 8bd2576327484b2a898ac41f422e02e4 P 933536aa412d4c8a90f9a9671cf38de6: Deleting consensus metadata
I20260809 18:52:00.662918 31990 ts_tablet_manager.cc:1939] T 8bd2576327484b2a898ac41f422e02e4 P 8f942b2c1bc0418d88e146376bc4c598: Deleting consensus metadata
I20260809 18:52:00.663239 31145 catalog_manager.cc:4841] TS 933536aa412d4c8a90f9a9671cf38de6 (127.23.218.194:35555): tablet 8bd2576327484b2a898ac41f422e02e4 (table test-table [id=00b3f02e518342ac85d19be047a9d369]) successfully deleted
I20260809 18:52:00.663318 31148 catalog_manager.cc:4841] TS 8f942b2c1bc0418d88e146376bc4c598 (127.23.218.195:41257): tablet 8bd2576327484b2a898ac41f422e02e4 (table test-table [id=00b3f02e518342ac85d19be047a9d369]) successfully deleted
W20260809 18:52:00.665158 31145 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:46477 (error 108)
I20260809 18:52:00.665438 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 31497
I20260809 18:52:00.665593 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.665477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46477 (local address 127.0.0.1:51726)
0809 18:52:00.665526 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.665564 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:46477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 18:52:00.681689 31145 catalog_manager.cc:4584] TS 1f68fb4e831747ec89aff3b778fc671f (127.23.218.193:46477): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8bd2576327484b2a898ac41f422e02e4: Network error: Client connection negotiation failed: client connection to 127.23.218.193:46477: connect: Connection refused (error 111)
I20260809 18:52:00.684955 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 31714
I20260809 18:52:00.699184 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 31105
2026-08-09T18:52:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2013 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T18:52:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:00Z Disabled control of system clock
I20260809 18:52:00.730950 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:40439
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:40439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 18:52:00.808143 32004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:00.808326 32004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:00.808346 32004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:00.809376 32004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:00.809406 32004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:00.809417 32004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:00.809428 32004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:00.809443 32004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:00.810715 32004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:40439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:40439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:00.810885 32004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:00.811094 32004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:00.815411 32019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:00.815521 32013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:00.815418 32015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.815583 32004 server_base.cc:1034] running on GCE node
I20260809 18:52:00.815814 32004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:00.816041 32004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:00.817174 32004 hybrid_clock.cc:648] HybridClock initialized: now 1786301520817160 us; error 30 us; skew 500 ppm
I20260809 18:52:00.818236 32004 webserver.cc:460] Webserver started at http://127.23.218.254:42451/ using document root <none> and password file <none>
I20260809 18:52:00.818464 32004 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:00.818508 32004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:00.818624 32004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:00.819553 32004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "a54d09d511b54691ae2f13b9b5bbbaf8"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "b0198a874b7b0f06ac90feeddc56e52e"
server_key_iv: "7a52dd8c6f1ea7c7aa73e8743a683d26"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.819861 32004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "a54d09d511b54691ae2f13b9b5bbbaf8"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "b0198a874b7b0f06ac90feeddc56e52e"
server_key_iv: "7a52dd8c6f1ea7c7aa73e8743a683d26"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.821130 32004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:52:00.821722 32029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.821851 32004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:00.821919 32004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "a54d09d511b54691ae2f13b9b5bbbaf8"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "b0198a874b7b0f06ac90feeddc56e52e"
server_key_iv: "7a52dd8c6f1ea7c7aa73e8743a683d26"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.821976 32004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:00.825169 32004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:00.825384 32004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:00.825490 32004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:00.828889 32004 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:40439
I20260809 18:52:00.828918 32113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:40439 every 8 connection(s)
I20260809 18:52:00.829178 32004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:00.829753 32114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:00.832173 32114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8: Bootstrap starting.
I20260809 18:52:00.832656 32114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:00.832887 32114 log.cc:824] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:00.833451 32114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8: No bootstrap required, opened a new log
I20260809 18:52:00.834547 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 32004
I20260809 18:52:00.834646 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:00.834825 24427 external_mini_cluster.cc:1428] Setting key 9a33a0ad6151252c86bad4c7f67ccf04
I20260809 18:52:00.835191 32114 raft_consensus.cc:348] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54d09d511b54691ae2f13b9b5bbbaf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 40439 } }
I20260809 18:52:00.835366 32114 raft_consensus.cc:374] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:00.835414 32114 raft_consensus.cc:729] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a54d09d511b54691ae2f13b9b5bbbaf8, State: Initialized, Role: FOLLOWER
I20260809 18:52:00.835534 32114 consensus_queue.cc:260] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54d09d511b54691ae2f13b9b5bbbaf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 40439 } }
I20260809 18:52:00.835609 32114 raft_consensus.cc:388] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:00.835644 32114 raft_consensus.cc:482] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:00.835685 32114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:00.836277 32114 raft_consensus.cc:504] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54d09d511b54691ae2f13b9b5bbbaf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 40439 } }
I20260809 18:52:00.836380 32114 leader_election.cc:302] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a54d09d511b54691ae2f13b9b5bbbaf8; no voters:
I20260809 18:52:00.836539 32114 leader_election.cc:288] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:00.836666 32120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:00.836771 32114 sys_catalog.cc:564] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:00.836812 32120 raft_consensus.cc:686] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [term 1 LEADER]: Becoming Leader. State: Replica: a54d09d511b54691ae2f13b9b5bbbaf8, State: Running, Role: LEADER
I20260809 18:52:00.836891 32120 consensus_queue.cc:237] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54d09d511b54691ae2f13b9b5bbbaf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 40439 } }
I20260809 18:52:00.837067 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.835272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43758 (local address 127.23.218.254:40439)
0809 18:52:00.835493 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.835501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.835531 (+ 30us) server_negotiation.cc:407] Connection header received
0809 18:52:00.835632 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.835640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.835682 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.835826 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:00.836347 (+ 521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:00.836350 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:00.836351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:00.836355 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:00.836398 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:00.836412 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:00.836492 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.836552 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.836608 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260809 18:52:00.837206 32120 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a54d09d511b54691ae2f13b9b5bbbaf8. Latest consensus state: current_term: 1 leader_uuid: "a54d09d511b54691ae2f13b9b5bbbaf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54d09d511b54691ae2f13b9b5bbbaf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 40439 } } }
I20260809 18:52:00.837277 32120 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:00.837913 32121 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a54d09d511b54691ae2f13b9b5bbbaf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54d09d511b54691ae2f13b9b5bbbaf8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 40439 } } }
I20260809 18:52:00.837996 32121 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:00.838073 32126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:00.838589 32126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:00.839883 32126 catalog_manager.cc:1347] Generated new cluster ID: 3276b277f5864af0a4cc1977b4787821
I20260809 18:52:00.839938 32126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:00.849602 32126 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:00.849738 32126 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:00.856335 32126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8: Generated new TSK 0
I20260809 18:52:00.856503 32126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:52:00.860651 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:40439
--builtin_ntp_servers=127.23.218.212:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 18:52:00.939028 32147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:00.939215 32147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:00.939245 32147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:00.940371 32147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:00.940409 32147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:00.940449 32147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:00.941807 32147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:40439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:00.942034 32147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:00.942301 32147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:00.947815 32157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:00.947825 32162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.947939 32147 server_base.cc:1034] running on GCE node
W20260809 18:52:00.948014 32156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:00.948287 32147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:00.948531 32147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:00.949689 32147 hybrid_clock.cc:648] HybridClock initialized: now 1786301520949672 us; error 33 us; skew 500 ppm
I20260809 18:52:00.951025 32147 webserver.cc:460] Webserver started at http://127.23.218.193:45185/ using document root <none> and password file <none>
I20260809 18:52:00.951299 32147 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:00.951359 32147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:00.951488 32147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:00.952811 32147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "4917626c67d14c8788954627ff78317b"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "79d2139cb401c73600192a877ef104fe"
server_key_iv: "909845d65e33eea23270457222d9bb4f"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.953217 32147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "4917626c67d14c8788954627ff78317b"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "79d2139cb401c73600192a877ef104fe"
server_key_iv: "909845d65e33eea23270457222d9bb4f"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.954818 32147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 18:52:00.955657 32169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.955823 32147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:00.955906 32147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "4917626c67d14c8788954627ff78317b"
format_stamp: "Formatted at 2026-08-09 18:52:00 on dist-test-slave-6zbq"
server_key: "79d2139cb401c73600192a877ef104fe"
server_key_iv: "909845d65e33eea23270457222d9bb4f"
server_key_version: "encryptionkey@0"
I20260809 18:52:00.955976 32147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:00.959071 32147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:00.959296 32147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:00.959409 32147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:00.959582 32147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:00.959851 32147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:00.959882 32147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.959906 32147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:00.959932 32147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:00.965018 32147 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:42915
I20260809 18:52:00.965080 32340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:42915 every 8 connection(s)
I20260809 18:52:00.965360 32147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:52:00.967854 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.965955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:52105 (local address 127.23.218.254:40439)
0809 18:52:00.965995 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:00.965997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:00.966215 (+ 218us) server_negotiation.cc:407] Connection header received
0809 18:52:00.966693 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:00.966694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:00.966700 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:00.966738 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:00.967685 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:00.967687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:00.967687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:00.967691 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:00.967705 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:00.967709 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:00.967748 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:00.967793 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 18:52:00.967804 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:52:00.968016 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:00.965944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:40439 (local address 127.23.218.193:52105)
0809 18:52:00.966111 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:00.966140 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:00.966604 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:00.966791 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:00.966794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:00.966801 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:00.967566 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:00.967566 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:00.967591 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:00.967617 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:00.967748 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:00.967750 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:00.967752 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:52:00.967799 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 18:52:00.967842 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260809 18:52:00.968446 32342 heartbeater.cc:347] Connected to a master server at 127.23.218.254:40439
I20260809 18:52:00.968528 32342 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:00.968631 32342 heartbeater.cc:511] Master 127.23.218.254:40439 requested a full tablet report, sending...
I20260809 18:52:00.969069 32055 ts_manager.cc:194] Registered new tserver with Master: 4917626c67d14c8788954627ff78317b (127.23.218.193:42915)
I20260809 18:52:00.975486 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 32147
I20260809 18:52:00.975564 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:52:00.975690 24427 external_mini_cluster.cc:1428] Setting key 53f839b69e2bed1c2a3300ad54db2ed4
I20260809 18:52:00.976428 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:40439
--builtin_ntp_servers=127.23.218.212:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 18:52:01.054991 32350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:01.055181 32350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:01.055212 32350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:01.056341 32350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:01.056380 32350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:01.056412 32350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:01.057717 32350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:40439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:01.057937 32350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:01.058244 32350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:01.062853 32359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:01.062861 32362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:01.063119 32350 server_base.cc:1034] running on GCE node
W20260809 18:52:01.062852 32357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:01.063426 32350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:01.063670 32350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:01.064810 32350 hybrid_clock.cc:648] HybridClock initialized: now 1786301521064802 us; error 27 us; skew 500 ppm
I20260809 18:52:01.065920 32350 webserver.cc:460] Webserver started at http://127.23.218.194:45055/ using document root <none> and password file <none>
I20260809 18:52:01.066125 32350 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:01.066170 32350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:01.066286 32350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:01.067250 32350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "070d318f17cf4cd19fa2730516558087"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "8a64ae2dd5aa185767f0aa4ac2f1690a"
server_key_iv: "8935702801285e9a2d18fd387c263511"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.067613 32350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "070d318f17cf4cd19fa2730516558087"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "8a64ae2dd5aa185767f0aa4ac2f1690a"
server_key_iv: "8935702801285e9a2d18fd387c263511"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.068864 32350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:01.069478 32375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.069638 32350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:01.069717 32350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "070d318f17cf4cd19fa2730516558087"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "8a64ae2dd5aa185767f0aa4ac2f1690a"
server_key_iv: "8935702801285e9a2d18fd387c263511"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.069775 32350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:01.072886 32350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:01.073107 32350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:01.073220 32350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:01.073395 32350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:01.073668 32350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:01.073705 32350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.073730 32350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:01.073755 32350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.079033 32350 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:40881
I20260809 18:52:01.079067 32558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:40881 every 8 connection(s)
I20260809 18:52:01.079324 32350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:52:01.080926 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 32350
I20260809 18:52:01.081014 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:52:01.081197 24427 external_mini_cluster.cc:1428] Setting key a04e8407ff80327d4dda8060e8db4320
I20260809 18:52:01.081954 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.079917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:60663 (local address 127.23.218.254:40439)
0809 18:52:01.079992 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.079995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.080152 (+ 157us) server_negotiation.cc:407] Connection header received
0809 18:52:01.080535 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.080536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.080541 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.080599 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.081725 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.081726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.081727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.081731 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.081747 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.081751 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.081814 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.081878 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.081892 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 18:52:01.082129 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.079895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:40439 (local address 127.23.218.194:60663)
0809 18:52:01.080053 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:01.080081 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:01.080467 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:01.080635 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:01.080638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:01.080646 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:01.081577 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:01.081578 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:01.081612 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:01.081642 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:01.081817 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:01.081820 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:01.081823 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:01.081888 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 18:52:01.081949 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260809 18:52:01.082255 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:40439
--builtin_ntp_servers=127.23.218.212:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 18:52:01.082562 32561 heartbeater.cc:347] Connected to a master server at 127.23.218.254:40439
I20260809 18:52:01.082643 32561 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:01.082769 32561 heartbeater.cc:511] Master 127.23.218.254:40439 requested a full tablet report, sending...
I20260809 18:52:01.083081 32055 ts_manager.cc:194] Registered new tserver with Master: 070d318f17cf4cd19fa2730516558087 (127.23.218.194:40881)
W20260809 18:52:01.160305 32565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:01.160480 32565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:01.160507 32565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:01.161628 32565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:01.161664 32565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:01.161696 32565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:01.163021 32565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:40439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:01.163239 32565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:01.163482 32565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:01.167973 32570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:01.168042 32573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:01.167985 32571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:01.167975 32565 server_base.cc:1034] running on GCE node
I20260809 18:52:01.168373 32565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:01.168582 32565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:01.169703 32565 hybrid_clock.cc:648] HybridClock initialized: now 1786301521169696 us; error 25 us; skew 500 ppm
I20260809 18:52:01.170792 32565 webserver.cc:460] Webserver started at http://127.23.218.195:39133/ using document root <none> and password file <none>
I20260809 18:52:01.170986 32565 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:01.171028 32565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:01.171137 32565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:01.172109 32565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "a24ff8ffd49043a49827bf330cbddfa8"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "342735370dcc2d4da459977ea103b2ee"
server_key_iv: "011cf4c4d9b44544e67969ccb0df9946"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.172415 32565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "a24ff8ffd49043a49827bf330cbddfa8"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "342735370dcc2d4da459977ea103b2ee"
server_key_iv: "011cf4c4d9b44544e67969ccb0df9946"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.173558 32565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:52:01.174131 32584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.174271 32565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.174373 32565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "a24ff8ffd49043a49827bf330cbddfa8"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "342735370dcc2d4da459977ea103b2ee"
server_key_iv: "011cf4c4d9b44544e67969ccb0df9946"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.174443 32565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:01.177515 32565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:01.177736 32565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:01.177848 32565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:01.178022 32565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:01.178294 32565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:01.178326 32565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.178367 32565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:01.178392 32565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.183744 32565 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:45563
I20260809 18:52:01.183801 32762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:45563 every 8 connection(s)
I20260809 18:52:01.184047 32565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:52:01.186462 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.184617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:51251 (local address 127.23.218.254:40439)
0809 18:52:01.184691 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.184693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.184851 (+ 158us) server_negotiation.cc:407] Connection header received
0809 18:52:01.185219 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.185220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.185225 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.185270 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.186266 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.186267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.186267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.186269 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.186278 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.186280 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.186314 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.186382 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.186397 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 18:52:01.186645 32765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.184589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:40439 (local address 127.23.218.195:51251)
0809 18:52:01.184753 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:01.184781 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:01.185148 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:01.185315 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:01.185318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:01.185325 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:01.186155 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:01.186155 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:01.186180 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:01.186206 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:01.186313 (+ 107us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:01.186315 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:01.186317 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:52:01.186392 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 18:52:01.186445 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
I20260809 18:52:01.186762 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 32565
I20260809 18:52:01.186858 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:52:01.187000 24427 external_mini_cluster.cc:1428] Setting key 1e0d1f1d27e607678e73bd548b2998c4
I20260809 18:52:01.187450 32763 heartbeater.cc:347] Connected to a master server at 127.23.218.254:40439
I20260809 18:52:01.187536 32763 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:01.187682 32763 heartbeater.cc:511] Master 127.23.218.254:40439 requested a full tablet report, sending...
I20260809 18:52:01.187999 32054 ts_manager.cc:194] Registered new tserver with Master: a24ff8ffd49043a49827bf330cbddfa8 (127.23.218.195:45563)
I20260809 18:52:01.198415 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:52:01.200425 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.199560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.23.218.254:40439)
0809 18:52:01.199620 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.199622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.199695 (+ 73us) server_negotiation.cc:407] Connection header received
0809 18:52:01.199742 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.199743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.199747 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.199781 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.200270 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.200271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.200271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.200274 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.200283 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.200286 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.200315 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.200355 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.200365 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 18:52:01.201912 32054 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:52:01.203395 32054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:01.206648 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.205126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:40881 (local address 127.0.0.1:53830)
0809 18:52:01.205283 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:01.205322 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:01.205379 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:01.205823 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:01.205824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:01.205827 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:01.206230 (+ 403us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:01.206230 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:01.206245 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:01.206266 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:01.206504 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:01.206511 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:01.206514 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:01.206576 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 18:52:01.206588 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260809 18:52:01.206712 310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.205217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.23.218.194:40881)
0809 18:52:01.205411 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.205416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.205424 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:01.205448 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.205450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.205458 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.205523 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.206327 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.206359 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.206360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.206365 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.206397 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.206422 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.206480 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.206607 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.206625 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":89,"threads_started":1}
I20260809 18:52:01.206895 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.205300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:42915 (local address 127.0.0.1:42900)
0809 18:52:01.205581 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:01.205591 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:01.205631 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:01.206122 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:01.206123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:01.206126 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:01.206534 (+ 408us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:01.206535 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:01.206540 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:01.206549 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:01.206788 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:01.206789 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:01.206790 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 18:52:01.206828 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 18:52:01.206839 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":62,"threads_started":1}
I20260809 18:52:01.207015 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.205412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.23.218.195:45563)
0809 18:52:01.205563 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.205567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.205629 (+ 62us) server_negotiation.cc:407] Connection header received
0809 18:52:01.205702 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.205705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.205712 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.205779 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.206562 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.206566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.206566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.206570 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.206603 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.206618 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.206676 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.206931 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.206946 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260809 18:52:01.207162 305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.205126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:45563 (local address 127.0.0.1:44266)
0809 18:52:01.205564 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:01.205574 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:01.205632 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:01.205787 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:01.205793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:01.205799 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:01.206380 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:01.206381 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:01.206388 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:01.206400 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:01.206823 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:01.206824 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:01.206825 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 18:52:01.206873 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 18:52:01.206885 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":4,"thread_start_us":43,"threads_started":1}
I20260809 18:52:01.207312 316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.205410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42900 (local address 127.23.218.193:42915)
0809 18:52:01.205740 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.205745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.205753 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:01.205778 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.205780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.205790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.205853 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.206624 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.206634 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.206634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.206638 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.206679 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.206705 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.206763 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.207050 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.207228 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":91,"threads_started":1}
I20260809 18:52:01.207309 32457 tablet_service.cc:1467] Processing CreateTablet for tablet d5d93485b61a40e5a212e295bcc5035a (DEFAULT_TABLE table=test-table [id=000683adca674cfab3aef1a53a5916cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:01.207659 32457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5d93485b61a40e5a212e295bcc5035a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:01.207826 32663 tablet_service.cc:1467] Processing CreateTablet for tablet d5d93485b61a40e5a212e295bcc5035a (DEFAULT_TABLE table=test-table [id=000683adca674cfab3aef1a53a5916cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:01.207859 32242 tablet_service.cc:1467] Processing CreateTablet for tablet d5d93485b61a40e5a212e295bcc5035a (DEFAULT_TABLE table=test-table [id=000683adca674cfab3aef1a53a5916cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:01.208117 32663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5d93485b61a40e5a212e295bcc5035a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:01.208209 32242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5d93485b61a40e5a212e295bcc5035a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:01.210637 318 tablet_bootstrap.cc:492] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b: Bootstrap starting.
I20260809 18:52:01.211205 319 tablet_bootstrap.cc:492] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: Bootstrap starting.
I20260809 18:52:01.211323 318 tablet_bootstrap.cc:654] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:01.211656 318 log.cc:824] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:01.211833 319 tablet_bootstrap.cc:654] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:01.212086 319 log.cc:824] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:01.212407 320 tablet_bootstrap.cc:492] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: Bootstrap starting.
I20260809 18:52:01.212432 318 tablet_bootstrap.cc:492] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b: No bootstrap required, opened a new log
I20260809 18:52:01.212492 318 ts_tablet_manager.cc:1397] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:52:01.212728 319 tablet_bootstrap.cc:492] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: No bootstrap required, opened a new log
I20260809 18:52:01.212795 319 ts_tablet_manager.cc:1397] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:52:01.213001 320 tablet_bootstrap.cc:654] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:01.213316 320 log.cc:824] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:01.214030 320 tablet_bootstrap.cc:492] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: No bootstrap required, opened a new log
I20260809 18:52:01.214099 320 ts_tablet_manager.cc:1397] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:52:01.214910 319 raft_consensus.cc:348] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } }
I20260809 18:52:01.215030 319 raft_consensus.cc:374] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:01.215062 319 raft_consensus.cc:729] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 070d318f17cf4cd19fa2730516558087, State: Initialized, Role: FOLLOWER
I20260809 18:52:01.215174 319 consensus_queue.cc:260] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } }
I20260809 18:52:01.215175 318 raft_consensus.cc:348] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } }
I20260809 18:52:01.215328 318 raft_consensus.cc:374] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:01.215371 318 raft_consensus.cc:729] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4917626c67d14c8788954627ff78317b, State: Initialized, Role: FOLLOWER
I20260809 18:52:01.215418 319 ts_tablet_manager.cc:1428] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 18:52:01.215482 318 consensus_queue.cc:260] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } }
I20260809 18:52:01.215528 32561 heartbeater.cc:503] Master 127.23.218.254:40439 was elected leader, sending a full tablet report...
I20260809 18:52:01.215764 32342 heartbeater.cc:503] Master 127.23.218.254:40439 was elected leader, sending a full tablet report...
W20260809 18:52:01.215879 32343 tablet.cc:2369] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:52:01.216028 318 ts_tablet_manager.cc:1428] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:52:01.216795 320 raft_consensus.cc:348] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } }
I20260809 18:52:01.216907 320 raft_consensus.cc:374] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:01.216924 320 raft_consensus.cc:729] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a24ff8ffd49043a49827bf330cbddfa8, State: Initialized, Role: FOLLOWER
I20260809 18:52:01.217020 320 consensus_queue.cc:260] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } }
I20260809 18:52:01.217234 320 ts_tablet_manager.cc:1428] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 18:52:01.217291 32763 heartbeater.cc:503] Master 127.23.218.254:40439 was elected leader, sending a full tablet report...
W20260809 18:52:01.329838 32563 tablet.cc:2369] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:52:01.434538 32764 tablet.cc:2369] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:52:01.513751 328 raft_consensus.cc:482] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:01.513885 328 raft_consensus.cc:504] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } }
I20260809 18:52:01.514240 328 leader_election.cc:288] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a24ff8ffd49043a49827bf330cbddfa8 (127.23.218.195:45563), 4917626c67d14c8788954627ff78317b (127.23.218.193:42915)
I20260809 18:52:01.515626 316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.514721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:42277 (local address 127.23.218.193:42915)
0809 18:52:01.514791 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.514794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.514804 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:52:01.514869 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.514870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.514875 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.514919 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.515421 (+ 502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.515423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.515423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.515427 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.515441 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.515445 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.515482 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.515543 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.515558 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 18:52:01.515638 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.514484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:45563 (local address 127.23.218.194:49215)
0809 18:52:01.514543 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:01.514556 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:01.514598 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:01.514763 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:01.514764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:01.514767 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:01.515196 (+ 429us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:01.515197 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:01.515206 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:01.515221 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:01.515504 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:01.515505 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:01.515506 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 18:52:01.515546 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 18:52:01.515562 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 18:52:01.515655 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.514562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:49215 (local address 127.23.218.195:45563)
0809 18:52:01.514617 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.514619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.514631 (+ 12us) server_negotiation.cc:407] Connection header received
0809 18:52:01.514662 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.514663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.514669 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.514744 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.515315 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.515317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.515317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.515322 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.515343 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.515348 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.515402 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.515573 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.515588 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 18:52:01.515641 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.514593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:42915 (local address 127.23.218.194:42277)
0809 18:52:01.514761 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:01.514769 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:01.514809 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:01.514931 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:01.514932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:01.514934 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:01.515326 (+ 392us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:01.515327 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:01.515333 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:01.515344 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:01.515479 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:01.515480 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:01.515480 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 18:52:01.515521 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 18:52:01.515536 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260809 18:52:01.515998 32273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d93485b61a40e5a212e295bcc5035a" candidate_uuid: "070d318f17cf4cd19fa2730516558087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4917626c67d14c8788954627ff78317b" is_pre_election: true
I20260809 18:52:01.516057 32700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d93485b61a40e5a212e295bcc5035a" candidate_uuid: "070d318f17cf4cd19fa2730516558087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24ff8ffd49043a49827bf330cbddfa8" is_pre_election: true
I20260809 18:52:01.516139 32273 raft_consensus.cc:2454] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 070d318f17cf4cd19fa2730516558087 in term 0.
I20260809 18:52:01.516209 32700 raft_consensus.cc:2454] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 070d318f17cf4cd19fa2730516558087 in term 0.
I20260809 18:52:01.516386 32383 leader_election.cc:302] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 070d318f17cf4cd19fa2730516558087, 4917626c67d14c8788954627ff78317b; no voters:
I20260809 18:52:01.516513 328 raft_consensus.cc:2781] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:01.516572 328 raft_consensus.cc:482] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:01.516600 328 raft_consensus.cc:3037] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:01.517272 328 raft_consensus.cc:504] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } }
I20260809 18:52:01.517455 328 leader_election.cc:288] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [CANDIDATE]: Term 1 election: Requested vote from peers a24ff8ffd49043a49827bf330cbddfa8 (127.23.218.195:45563), 4917626c67d14c8788954627ff78317b (127.23.218.193:42915)
I20260809 18:52:01.517500 32700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d93485b61a40e5a212e295bcc5035a" candidate_uuid: "070d318f17cf4cd19fa2730516558087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a24ff8ffd49043a49827bf330cbddfa8"
I20260809 18:52:01.517576 32700 raft_consensus.cc:3037] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:01.517593 32273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5d93485b61a40e5a212e295bcc5035a" candidate_uuid: "070d318f17cf4cd19fa2730516558087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4917626c67d14c8788954627ff78317b"
I20260809 18:52:01.517663 32273 raft_consensus.cc:3037] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:01.518321 32273 raft_consensus.cc:2454] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 070d318f17cf4cd19fa2730516558087 in term 1.
I20260809 18:52:01.518462 32700 raft_consensus.cc:2454] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 070d318f17cf4cd19fa2730516558087 in term 1.
I20260809 18:52:01.518481 32383 leader_election.cc:302] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 070d318f17cf4cd19fa2730516558087, 4917626c67d14c8788954627ff78317b; no voters:
I20260809 18:52:01.518558 328 raft_consensus.cc:2781] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:01.518973 328 raft_consensus.cc:686] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 1 LEADER]: Becoming Leader. State: Replica: 070d318f17cf4cd19fa2730516558087, State: Running, Role: LEADER
I20260809 18:52:01.519061 328 consensus_queue.cc:237] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } }
I20260809 18:52:01.519886 32055 catalog_manager.cc:5495] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 reported cstate change: term changed from 0 to 1, leader changed from <none> to 070d318f17cf4cd19fa2730516558087 (127.23.218.194). New cstate: current_term: 1 leader_uuid: "070d318f17cf4cd19fa2730516558087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "070d318f17cf4cd19fa2730516558087" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40881 } health_report { overall_health: HEALTHY } } }
I20260809 18:52:01.557142 310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.556276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53836 (local address 127.23.218.194:40881)
0809 18:52:01.556326 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.556328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.556336 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:01.556399 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.556400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.556415 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.556466 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.556929 (+ 463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.556930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.556931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.556935 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.556950 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.556954 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.557002 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.557054 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.557066 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 18:52:01.558899 32700 raft_consensus.cc:1261] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [term 1 FOLLOWER]: Refusing update from remote peer 070d318f17cf4cd19fa2730516558087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:01.558899 32273 raft_consensus.cc:1261] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [term 1 FOLLOWER]: Refusing update from remote peer 070d318f17cf4cd19fa2730516558087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:01.559234 335 consensus_queue.cc:1035] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a24ff8ffd49043a49827bf330cbddfa8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 45563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:01.559365 328 consensus_queue.cc:1035] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4917626c67d14c8788954627ff78317b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 42915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:01.561702 343 mvcc.cc:205] Tried to move back new op lower bound from 7316691032302419968 to 7316691032142589952. Current Snapshot: MvccSnapshot[applied={T|T < 7316691032302419968}]
I20260809 18:52:01.563735 32054 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:01.563810 32054 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:01.564422 32054 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a54d09d511b54691ae2f13b9b5bbbaf8: Sending DeleteTablet for 3 replicas of tablet d5d93485b61a40e5a212e295bcc5035a
I20260809 18:52:01.564754 32457 tablet_service.cc:1514] Processing DeleteTablet for tablet d5d93485b61a40e5a212e295bcc5035a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:01 UTC) from {username='slave'} at 127.0.0.1:53830
I20260809 18:52:01.564836 32663 tablet_service.cc:1514] Processing DeleteTablet for tablet d5d93485b61a40e5a212e295bcc5035a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:01 UTC) from {username='slave'} at 127.0.0.1:44266
I20260809 18:52:01.564863 32242 tablet_service.cc:1514] Processing DeleteTablet for tablet d5d93485b61a40e5a212e295bcc5035a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:01 UTC) from {username='slave'} at 127.0.0.1:42900
I20260809 18:52:01.565179 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 32147
I20260809 18:52:01.565297 365 tablet_replica.cc:330] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b: stopping tablet replica
I20260809 18:52:01.565394 365 raft_consensus.cc:2229] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:01.565495 365 raft_consensus.cc:2258] T d5d93485b61a40e5a212e295bcc5035a P 4917626c67d14c8788954627ff78317b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:01.565658 363 tablet_replica.cc:330] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: stopping tablet replica
I20260809 18:52:01.565743 363 raft_consensus.cc:2229] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:52:01.565822 363 raft_consensus.cc:2258] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:01.571548 364 tablet_replica.cc:330] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: stopping tablet replica
I20260809 18:52:01.571652 364 raft_consensus.cc:2229] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:01.571739 364 raft_consensus.cc:2258] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:01.572513 364 ts_tablet_manager.cc:1905] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:01.572522 363 ts_tablet_manager.cc:1905] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:01.574116 364 ts_tablet_manager.cc:1918] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:01.574196 364 log.cc:1196] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/d5d93485b61a40e5a212e295bcc5035a
I20260809 18:52:01.574469 364 ts_tablet_manager.cc:1939] T d5d93485b61a40e5a212e295bcc5035a P a24ff8ffd49043a49827bf330cbddfa8: Deleting consensus metadata
I20260809 18:52:01.574887 32036 catalog_manager.cc:4841] TS a24ff8ffd49043a49827bf330cbddfa8 (127.23.218.195:45563): tablet d5d93485b61a40e5a212e295bcc5035a (table test-table [id=000683adca674cfab3aef1a53a5916cd]) successfully deleted
I20260809 18:52:01.575147 363 ts_tablet_manager.cc:1918] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:01.575296 363 log.cc:1196] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/d5d93485b61a40e5a212e295bcc5035a
I20260809 18:52:01.575704 363 ts_tablet_manager.cc:1939] T d5d93485b61a40e5a212e295bcc5035a P 070d318f17cf4cd19fa2730516558087: Deleting consensus metadata
I20260809 18:52:01.587800 32034 catalog_manager.cc:4841] TS 070d318f17cf4cd19fa2730516558087 (127.23.218.194:40881): tablet d5d93485b61a40e5a212e295bcc5035a (table test-table [id=000683adca674cfab3aef1a53a5916cd]) successfully deleted
W20260809 18:52:01.594436 32036 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:42915 (error 108)
I20260809 18:52:01.594913 305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.594795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:42915 (local address 127.0.0.1:42916)
0809 18:52:01.594844 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:01.594882 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:42915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 18:52:01.595031 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 32350
W20260809 18:52:01.595034 32036 catalog_manager.cc:4584] TS 4917626c67d14c8788954627ff78317b (127.23.218.193:42915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5d93485b61a40e5a212e295bcc5035a: Network error: Client connection negotiation failed: client connection to 127.23.218.193:42915: connect: Connection refused (error 111)
I20260809 18:52:01.603701 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 32565
I20260809 18:52:01.617108 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 32004
2026-08-09T18:52:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (918 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 18:52:01.642765 24427 mini_oidc.cc:152] Starting JWKS server
I20260809 18:52:01.643899 24427 webserver.cc:460] Webserver started at https://127.0.0.1:35387/ using document root <none> and password file <none>
I20260809 18:52:01.643975 24427 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 18:52:01.644101 24427 webserver.cc:460] Webserver started at http://127.0.0.1:46265/ using document root <none> and password file <none>
2026-08-09T18:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:01Z Disabled control of system clock
I20260809 18:52:01.651822 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:32901
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:32901
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35387/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/testchainca.cert with env {}
W20260809 18:52:01.729153 376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:01.729327 376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:01.729347 376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:01.730490 376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:01.730527 376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:01.730540 376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:01.730552 376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:01.730576 376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 18:52:01.730599 376 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 18:52:01.730619 376 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35387/jwks.json
I20260809 18:52:01.732096 376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:32901
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:32901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35387/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:01.732309 376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:01.732489 376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:01.736907 393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:01.736915 390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:01.736920 385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:01.737028 376 server_base.cc:1034] running on GCE node
I20260809 18:52:01.737324 376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:01.737581 376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:01.738718 376 hybrid_clock.cc:648] HybridClock initialized: now 1786301521738704 us; error 26 us; skew 500 ppm
I20260809 18:52:01.739775 376 webserver.cc:460] Webserver started at http://127.23.218.254:45629/ using document root <none> and password file <none>
I20260809 18:52:01.739966 376 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:01.740013 376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:01.740127 376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:01.741053 376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "980f1fb5714845008e2c9cd485dfe1fa"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "8c20219081203b7174c2723944435687"
server_key_iv: "0ece169e591a209a434be19f557e3d7d"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.741376 376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "980f1fb5714845008e2c9cd485dfe1fa"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "8c20219081203b7174c2723944435687"
server_key_iv: "0ece169e591a209a434be19f557e3d7d"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.742527 376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:01.743140 399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.743285 376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:01.743366 376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "980f1fb5714845008e2c9cd485dfe1fa"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "8c20219081203b7174c2723944435687"
server_key_iv: "0ece169e591a209a434be19f557e3d7d"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.743429 376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:01.750027 376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:01.750259 376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:01.750415 376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:01.753824 376 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:32901
I20260809 18:52:01.753854 479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:32901 every 8 connection(s)
I20260809 18:52:01.754123 376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:01.754968 480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:01.755534 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 376
I20260809 18:52:01.755642 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:01.755788 24427 external_mini_cluster.cc:1428] Setting key a60a0bbaab0a115b5ee858136e697cad
I20260809 18:52:01.757701 480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa: Bootstrap starting.
I20260809 18:52:01.758378 480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:01.758664 480 log.cc:824] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:01.759279 480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa: No bootstrap required, opened a new log
I20260809 18:52:01.761240 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.756207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40710 (local address 127.23.218.254:32901)
0809 18:52:01.756397 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.756405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.756437 (+ 32us) server_negotiation.cc:407] Connection header received
0809 18:52:01.756913 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.756923 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.757002 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.757133 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.757285 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:01.758573 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:01.759320 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:01.759900 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:01.760021 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:01.760564 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.760568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.760568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.760752 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.760789 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.760803 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.760867 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.760931 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.761010 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260809 18:52:01.761521 480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980f1fb5714845008e2c9cd485dfe1fa" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 32901 } }
I20260809 18:52:01.761636 480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:01.761656 480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 980f1fb5714845008e2c9cd485dfe1fa, State: Initialized, Role: FOLLOWER
I20260809 18:52:01.761915 480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980f1fb5714845008e2c9cd485dfe1fa" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 32901 } }
I20260809 18:52:01.762001 480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:01.762032 480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:01.762079 480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:01.762292 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:52:01.762693 480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980f1fb5714845008e2c9cd485dfe1fa" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 32901 } }
I20260809 18:52:01.762789 480 leader_election.cc:302] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 980f1fb5714845008e2c9cd485dfe1fa; no voters:
I20260809 18:52:01.762941 480 leader_election.cc:288] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:01.762990 486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:01.763139 486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [term 1 LEADER]: Becoming Leader. State: Replica: 980f1fb5714845008e2c9cd485dfe1fa, State: Running, Role: LEADER
I20260809 18:52:01.763239 486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980f1fb5714845008e2c9cd485dfe1fa" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 32901 } }
I20260809 18:52:01.763363 480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:01.763538 486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 980f1fb5714845008e2c9cd485dfe1fa. Latest consensus state: current_term: 1 leader_uuid: "980f1fb5714845008e2c9cd485dfe1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980f1fb5714845008e2c9cd485dfe1fa" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 32901 } } }
I20260809 18:52:01.763621 486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:01.763548 487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "980f1fb5714845008e2c9cd485dfe1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "980f1fb5714845008e2c9cd485dfe1fa" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 32901 } } }
I20260809 18:52:01.763777 487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:01.763962 493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:01.764458 493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:01.769313 493 catalog_manager.cc:1347] Generated new cluster ID: de640cf61b4b43e3b645c8bfc180817f
I20260809 18:52:01.769361 493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:01.785485 493 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:01.785908 493 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:01.795846 493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 980f1fb5714845008e2c9cd485dfe1fa: Generated new TSK 0
I20260809 18:52:01.795992 493 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 18:52:01.825542 530 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 18:52:01.825714 530 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 18:52:01.825735 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.811269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.23.218.254:32901)
0809 18:52:01.811307 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.811308 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.811410 (+ 102us) server_negotiation.cc:407] Connection header received
0809 18:52:01.811499 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.811500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.811508 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.811560 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0809 18:52:01.811709 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:01.812148 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:01.813216 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:01.813396 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:01.813436 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:01.813653 (+ 217us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 18:52:01.825534 (+ 11881us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35387/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 18:52:01.825675 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40720: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35387/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 18:52:01.825788 484 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40720: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35387/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 18:52:01.826314 529 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:01.811182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:32901 (local address 127.0.0.1:40720)
0809 18:52:01.811343 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:01.811353 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:01.811422 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:01.811566 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:01.811566 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:01.811571 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0809 18:52:01.811651 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:01.811653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:01.812209 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:01.812210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:01.813144 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:01.813146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:01.813468 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:01.813469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:01.813533 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:01.813554 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 18:52:01.825714 (+ 12160us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35387/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 18:52:01.826007 (+ 293us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.218.254:32901: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35387/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.23.218.254:32901: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35387/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 18:52:01.826774 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 376
2026-08-09T18:52:01Z chronyd exiting
I20260809 18:52:01.866453 24427 test_util.cc:175] -----------------------------------------------
I20260809 18:52:01.866518 24427 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786301499543878-24427-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (224 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 18:52:01.867254 24427 mini_oidc.cc:152] Starting JWKS server
I20260809 18:52:01.867895 24427 webserver.cc:460] Webserver started at https://127.0.0.1:36367/ using document root <none> and password file <none>
I20260809 18:52:01.867937 24427 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 18:52:01.868070 24427 webserver.cc:460] Webserver started at http://127.0.0.1:42181/ using document root <none> and password file <none>
2026-08-09T18:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:01Z Disabled control of system clock
I20260809 18:52:01.875526 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:44143
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:44143
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36367/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/testchainca.cert with env {}
W20260809 18:52:01.952311 538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:01.952499 538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:01.952528 538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:01.953629 538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:01.953668 538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:01.953685 538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:01.953701 538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:01.953716 538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 18:52:01.953747 538 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 18:52:01.953771 538 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36367/jwks.json
I20260809 18:52:01.955264 538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:44143
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:44143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36367/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:01.955490 538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:01.955679 538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:01.960039 553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:01.960026 549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:01.960021 551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:01.960186 538 server_base.cc:1034] running on GCE node
I20260809 18:52:01.960392 538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:01.960649 538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:01.961781 538 hybrid_clock.cc:648] HybridClock initialized: now 1786301521961768 us; error 28 us; skew 500 ppm
I20260809 18:52:01.962865 538 webserver.cc:460] Webserver started at http://127.23.218.254:42711/ using document root <none> and password file <none>
I20260809 18:52:01.963054 538 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:01.963099 538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:01.963218 538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:01.964162 538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "8114338a2a094dc0a788e90d9b0b2d81"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "8cf4faeed6f13bec1e8b815d9e0ad146"
server_key_iv: "a9817df9e699d12a179a1de242fcbab7"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.964466 538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "8114338a2a094dc0a788e90d9b0b2d81"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "8cf4faeed6f13bec1e8b815d9e0ad146"
server_key_iv: "a9817df9e699d12a179a1de242fcbab7"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.965564 538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:52:01.966284 563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.966437 538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:01.966513 538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "8114338a2a094dc0a788e90d9b0b2d81"
format_stamp: "Formatted at 2026-08-09 18:52:01 on dist-test-slave-6zbq"
server_key: "8cf4faeed6f13bec1e8b815d9e0ad146"
server_key_iv: "a9817df9e699d12a179a1de242fcbab7"
server_key_version: "encryptionkey@0"
I20260809 18:52:01.966578 538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:01.972884 538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:01.973111 538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:01.973223 538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:01.976790 538 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:44143
I20260809 18:52:01.976845 644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:44143 every 8 connection(s)
I20260809 18:52:01.977118 538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:01.977705 647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:01.979005 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 538
I20260809 18:52:01.979099 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:01.979256 24427 external_mini_cluster.cc:1428] Setting key a6ded0c4fcdb11c634a1ab77b420fb6c
I20260809 18:52:01.980474 647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81: Bootstrap starting.
I20260809 18:52:01.981163 647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:01.981465 647 log.cc:824] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:01.982143 647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81: No bootstrap required, opened a new log
I20260809 18:52:01.984236 647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114338a2a094dc0a788e90d9b0b2d81" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44143 } }
I20260809 18:52:01.984396 647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:01.984431 647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8114338a2a094dc0a788e90d9b0b2d81, State: Initialized, Role: FOLLOWER
I20260809 18:52:01.984556 647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114338a2a094dc0a788e90d9b0b2d81" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44143 } }
I20260809 18:52:01.984639 647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:01.984669 647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:01.984706 647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:01.984870 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:01.979665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.23.218.254:44143)
0809 18:52:01.979867 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:01.979876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:01.979906 (+ 30us) server_negotiation.cc:407] Connection header received
0809 18:52:01.980365 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:01.980374 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:01.980446 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:01.980587 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:01.980789 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:01.982051 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:01.982846 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:01.983460 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:01.983592 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:01.984162 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:01.984166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:01.984167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:01.984360 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:01.984402 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:01.984417 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:01.984480 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:01.984599 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 18:52:01.984679 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260809 18:52:01.985257 647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114338a2a094dc0a788e90d9b0b2d81" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44143 } }
I20260809 18:52:01.985356 647 leader_election.cc:302] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8114338a2a094dc0a788e90d9b0b2d81; no voters:
I20260809 18:52:01.985507 647 leader_election.cc:288] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:01.985558 654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:01.985698 654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [term 1 LEADER]: Becoming Leader. State: Replica: 8114338a2a094dc0a788e90d9b0b2d81, State: Running, Role: LEADER
I20260809 18:52:01.985756 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:52:01.985751 647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:01.986380 654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114338a2a094dc0a788e90d9b0b2d81" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44143 } }
I20260809 18:52:01.986894 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8114338a2a094dc0a788e90d9b0b2d81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114338a2a094dc0a788e90d9b0b2d81" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44143 } } }
I20260809 18:52:01.986981 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:01.987342 669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:01.987377 657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8114338a2a094dc0a788e90d9b0b2d81. Latest consensus state: current_term: 1 leader_uuid: "8114338a2a094dc0a788e90d9b0b2d81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114338a2a094dc0a788e90d9b0b2d81" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44143 } } }
I20260809 18:52:01.987519 657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:01.988060 669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:01.989517 669 catalog_manager.cc:1347] Generated new cluster ID: 00deb72d0ba442b5abe4c11fdc88a7eb
I20260809 18:52:01.989569 669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:01.998561 669 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:01.999004 669 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:02.008607 669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8114338a2a094dc0a788e90d9b0b2d81: Generated new TSK 0
I20260809 18:52:02.008761 669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:52:02.027081 652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:02.013665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.23.218.254:44143)
0809 18:52:02.013722 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:02.013724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:02.013802 (+ 78us) server_negotiation.cc:407] Connection header received
0809 18:52:02.013861 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:02.013862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:02.013873 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:02.013927 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0809 18:52:02.014053 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.014577 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.015313 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.015497 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.015549 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:02.015709 (+ 160us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 18:52:02.026904 (+ 11195us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36367/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 18:52:02.027021 (+ 117us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47632: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36367/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 18:52:02.027138 652 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47632: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36367/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 18:52:02.027182 691 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:02.013612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:44143 (local address 127.0.0.1:47632)
0809 18:52:02.013741 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:02.013749 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:02.013806 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:02.013931 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:02.013931 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:02.013933 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 18:52:02.013997 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:02.013999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.014646 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.014647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:02.015240 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:02.015243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.015557 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.015559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:02.015624 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:02.015629 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 18:52:02.026991 (+ 11362us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36367/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 18:52:02.027076 (+ 85us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.218.254:44143: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36367/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
E20260809 18:52:02.027347 693 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 18:52:02.027391 693 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 18:52:02.027627 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 538
2026-08-09T18:52:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (205 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 18:52:02.072369 24427 mini_oidc.cc:152] Starting JWKS server
I20260809 18:52:02.072978 24427 webserver.cc:460] Webserver started at https://127.0.0.1:41645/ using document root <none> and password file <none>
I20260809 18:52:02.073017 24427 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 18:52:02.073143 24427 webserver.cc:460] Webserver started at http://127.0.0.1:43911/ using document root <none> and password file <none>
2026-08-09T18:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:02Z Disabled control of system clock
I20260809 18:52:02.080626 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:35069
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:35069
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41645/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 18:52:02.158114 707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:02.158313 707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:02.158360 707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:02.159471 707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:02.159507 707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:02.159523 707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:02.159539 707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:02.159554 707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 18:52:02.159570 707 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 18:52:02.159592 707 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41645/jwks.json
I20260809 18:52:02.160912 707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:35069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:35069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41645/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:02.161161 707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:02.161422 707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:02.165705 717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:02.165727 721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:02.165727 714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:02.165890 707 server_base.cc:1034] running on GCE node
I20260809 18:52:02.166097 707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:02.166407 707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:02.167547 707 hybrid_clock.cc:648] HybridClock initialized: now 1786301522167534 us; error 29 us; skew 500 ppm
I20260809 18:52:02.168637 707 webserver.cc:460] Webserver started at http://127.23.218.254:45025/ using document root <none> and password file <none>
I20260809 18:52:02.168853 707 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:02.168900 707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:02.169024 707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:02.169976 707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "e8fbc0349120442fb67188306be563c6"
format_stamp: "Formatted at 2026-08-09 18:52:02 on dist-test-slave-6zbq"
server_key: "8af0e610e7e4fbf60770dca889a17d59"
server_key_iv: "3b598c5b32ab415c9b1b18394dd16948"
server_key_version: "encryptionkey@0"
I20260809 18:52:02.170295 707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "e8fbc0349120442fb67188306be563c6"
format_stamp: "Formatted at 2026-08-09 18:52:02 on dist-test-slave-6zbq"
server_key: "8af0e610e7e4fbf60770dca889a17d59"
server_key_iv: "3b598c5b32ab415c9b1b18394dd16948"
server_key_version: "encryptionkey@0"
I20260809 18:52:02.171485 707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:52:02.172128 730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:02.172281 707 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:02.172361 707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "e8fbc0349120442fb67188306be563c6"
format_stamp: "Formatted at 2026-08-09 18:52:02 on dist-test-slave-6zbq"
server_key: "8af0e610e7e4fbf60770dca889a17d59"
server_key_iv: "3b598c5b32ab415c9b1b18394dd16948"
server_key_version: "encryptionkey@0"
I20260809 18:52:02.172426 707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:02.192828 707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:02.193097 707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:02.193219 707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:02.196832 707 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:35069
I20260809 18:52:02.196866 809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:35069 every 8 connection(s)
I20260809 18:52:02.197158 707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:02.197775 810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:02.200246 810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6: Bootstrap starting.
I20260809 18:52:02.200753 810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:02.200969 810 log.cc:824] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:02.201553 810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6: No bootstrap required, opened a new log
I20260809 18:52:02.203110 810 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fbc0349120442fb67188306be563c6" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35069 } }
I20260809 18:52:02.203241 810 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:02.203263 810 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8fbc0349120442fb67188306be563c6, State: Initialized, Role: FOLLOWER
I20260809 18:52:02.203364 810 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fbc0349120442fb67188306be563c6" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35069 } }
I20260809 18:52:02.203413 810 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:02.203431 810 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:02.203460 810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:02.203917 810 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fbc0349120442fb67188306be563c6" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35069 } }
I20260809 18:52:02.203999 810 leader_election.cc:302] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8fbc0349120442fb67188306be563c6; no voters:
I20260809 18:52:02.204113 810 leader_election.cc:288] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:02.204164 816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:02.204279 816 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [term 1 LEADER]: Becoming Leader. State: Replica: e8fbc0349120442fb67188306be563c6, State: Running, Role: LEADER
I20260809 18:52:02.204357 810 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:02.204381 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 707
I20260809 18:52:02.204383 816 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fbc0349120442fb67188306be563c6" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35069 } }
I20260809 18:52:02.204473 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:02.204641 24427 external_mini_cluster.cc:1428] Setting key a0dacc3acdced1dc2d5af682a38b5773
I20260809 18:52:02.204741 816 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8fbc0349120442fb67188306be563c6. Latest consensus state: current_term: 1 leader_uuid: "e8fbc0349120442fb67188306be563c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fbc0349120442fb67188306be563c6" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35069 } } }
I20260809 18:52:02.204809 816 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:02.204946 817 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8fbc0349120442fb67188306be563c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fbc0349120442fb67188306be563c6" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35069 } } }
I20260809 18:52:02.204991 817 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:02.205000 828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:02.205503 828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:02.207300 828 catalog_manager.cc:1347] Generated new cluster ID: 44c4aefb7dc04283b2850fcc70dbdedb
I20260809 18:52:02.207352 828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:02.210762 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:02.205445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58758 (local address 127.23.218.254:35069)
0809 18:52:02.205924 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:02.205933 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:02.205964 (+ 31us) server_negotiation.cc:407] Connection header received
0809 18:52:02.206178 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:02.206186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:02.206259 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:02.206405 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:02.206613 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.207734 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.208786 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.209349 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.209447 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:02.210077 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:02.210081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:02.210082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:02.210268 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:02.210307 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:02.210320 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:02.210409 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:02.210486 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 18:52:02.210558 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":70,"threads_started":1}
I20260809 18:52:02.211560 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:52:02.223860 828 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:02.224531 828 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:02.232777 828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8fbc0349120442fb67188306be563c6: Generated new TSK 0
I20260809 18:52:02.232935 828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:52:02.243307 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:02.240311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.23.218.254:35069)
0809 18:52:02.240351 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:02.240354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:02.240394 (+ 40us) server_negotiation.cc:407] Connection header received
0809 18:52:02.240464 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:02.240465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:02.240477 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:02.240535 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0809 18:52:02.240657 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.241147 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.241895 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.242096 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.242149 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:02.242314 (+ 165us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 18:52:02.243117 (+ 803us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41645/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 18:52:02.243246 (+ 129us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58770: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41645/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 18:52:02.243360 837 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58770: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41645/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 18:52:02.243536 855 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:02.240207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:35069 (local address 127.0.0.1:58770)
0809 18:52:02.240344 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:02.240354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:02.240411 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:02.240538 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:02.240539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:02.240540 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0809 18:52:02.240591 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:02.240593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.241225 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.241225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:02.241821 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:02.241823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.242171 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.242172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:02.242230 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:02.242234 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 18:52:02.243212 (+ 978us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41645/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 18:52:02.243316 (+ 104us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.218.254:35069: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41645/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260809 18:52:02.243886 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 707
E20260809 18:52:02.268018 857 webserver.cc:533] Webserver: sslize failed:
E20260809 18:52:02.268088 857 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T18:52:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T18:52:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:02Z Disabled control of system clock
I20260809 18:52:02.299293 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:41897
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:41897
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/testchainca.cert with env {}
W20260809 18:52:02.382805 862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:02.382979 862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:02.383000 862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:02.384284 862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:02.384327 862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:02.384338 862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:02.384351 862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:02.384361 862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 18:52:02.384377 862 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 18:52:02.384398 862 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 18:52:02.386418 862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:41897
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:02.386648 862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:02.386842 862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:02.391263 868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:02.391268 872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:02.391301 862 server_base.cc:1034] running on GCE node
W20260809 18:52:02.391268 870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:02.391633 862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:02.391881 862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:02.393014 862 hybrid_clock.cc:648] HybridClock initialized: now 1786301522393000 us; error 27 us; skew 500 ppm
I20260809 18:52:02.394083 862 webserver.cc:460] Webserver started at http://127.23.218.254:40747/ using document root <none> and password file <none>
I20260809 18:52:02.394304 862 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:02.394392 862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:02.394508 862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:02.395444 862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "2b9979a7167d4a81bb42f37f12783543"
format_stamp: "Formatted at 2026-08-09 18:52:02 on dist-test-slave-6zbq"
server_key: "c066a76f74488f9fada474eea7c31702"
server_key_iv: "2d04aaadb0760c258bdbecab3a0ddb5a"
server_key_version: "encryptionkey@0"
I20260809 18:52:02.395750 862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "2b9979a7167d4a81bb42f37f12783543"
format_stamp: "Formatted at 2026-08-09 18:52:02 on dist-test-slave-6zbq"
server_key: "c066a76f74488f9fada474eea7c31702"
server_key_iv: "2d04aaadb0760c258bdbecab3a0ddb5a"
server_key_version: "encryptionkey@0"
I20260809 18:52:02.396878 862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:02.397526 878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:02.397696 862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:02.397778 862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "2b9979a7167d4a81bb42f37f12783543"
format_stamp: "Formatted at 2026-08-09 18:52:02 on dist-test-slave-6zbq"
server_key: "c066a76f74488f9fada474eea7c31702"
server_key_iv: "2d04aaadb0760c258bdbecab3a0ddb5a"
server_key_version: "encryptionkey@0"
I20260809 18:52:02.397837 862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:02.409271 862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:02.409484 862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:02.409595 862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:02.413091 862 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:41897
I20260809 18:52:02.413118 952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:41897 every 8 connection(s)
I20260809 18:52:02.413403 862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:02.414270 953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:02.416306 953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543: Bootstrap starting.
I20260809 18:52:02.416815 953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:02.417057 953 log.cc:824] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:02.417778 953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543: No bootstrap required, opened a new log
I20260809 18:52:02.419314 953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9979a7167d4a81bb42f37f12783543" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41897 } }
I20260809 18:52:02.419438 953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:02.419469 953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9979a7167d4a81bb42f37f12783543, State: Initialized, Role: FOLLOWER
I20260809 18:52:02.419596 953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9979a7167d4a81bb42f37f12783543" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41897 } }
I20260809 18:52:02.419654 953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:02.419673 953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:02.419713 953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:02.420183 953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9979a7167d4a81bb42f37f12783543" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41897 } }
I20260809 18:52:02.420270 953 leader_election.cc:302] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b9979a7167d4a81bb42f37f12783543; no voters:
I20260809 18:52:02.420413 953 leader_election.cc:288] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:02.420447 958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:02.420573 958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [term 1 LEADER]: Becoming Leader. State: Replica: 2b9979a7167d4a81bb42f37f12783543, State: Running, Role: LEADER
I20260809 18:52:02.420642 953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:02.420668 958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9979a7167d4a81bb42f37f12783543" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41897 } }
I20260809 18:52:02.420984 959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b9979a7167d4a81bb42f37f12783543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9979a7167d4a81bb42f37f12783543" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41897 } } }
I20260809 18:52:02.421082 959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:02.421088 960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b9979a7167d4a81bb42f37f12783543. Latest consensus state: current_term: 1 leader_uuid: "2b9979a7167d4a81bb42f37f12783543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9979a7167d4a81bb42f37f12783543" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41897 } } }
I20260809 18:52:02.421166 960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:02.421306 968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:02.421787 968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:02.423156 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 862
I20260809 18:52:02.423252 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:02.423287 968 catalog_manager.cc:1347] Generated new cluster ID: 0bd7260d3c19414db13db2a4b13740fa
I20260809 18:52:02.423322 968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:02.423429 24427 external_mini_cluster.cc:1428] Setting key ea4c8d455e62a5b5878e5ec48de93d28
I20260809 18:52:02.428998 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:02.423927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.23.218.254:41897)
0809 18:52:02.424114 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:02.424122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:02.424164 (+ 42us) server_negotiation.cc:407] Connection header received
0809 18:52:02.424281 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:02.424288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:02.424352 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:02.424491 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:02.424648 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.426031 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.427033 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.427618 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.427727 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:02.428361 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:02.428365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:02.428365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:02.428550 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:02.428588 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:02.428601 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:02.428657 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:02.428750 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 18:52:02.428819 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260809 18:52:02.429901 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:52:02.430496 968 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:02.431052 968 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:02.436210 968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b9979a7167d4a81bb42f37f12783543: Generated new TSK 0
I20260809 18:52:02.436367 968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:52:03.309319 986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:02.441393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34578 (local address 127.23.218.254:41897)
0809 18:52:02.441441 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:02.441442 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:02.441549 (+ 107us) server_negotiation.cc:407] Connection header received
0809 18:52:02.441611 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:02.441611 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:02.441622 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:02.441678 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0809 18:52:02.441784 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.442242 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.443011 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.443205 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.443263 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:02.443407 (+ 144us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 18:52:03.309055 (+865648us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 18:52:03.309243 (+ 188us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34578: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":28}
W20260809 18:52:03.309404 986 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34578: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 18:52:03.309435 998 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:02.441331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41897 (local address 127.0.0.1:34578)
0809 18:52:02.441489 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:02.441497 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:02.441549 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:02.441675 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:02.441676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:02.441677 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0809 18:52:02.441725 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:02.441726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.442324 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.442325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:02.442940 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:02.442942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:02.443270 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:02.443272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:02.443325 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:02.443330 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 18:52:03.309201 (+865871us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 18:52:03.309338 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.218.254:41897: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260809 18:52:03.309841 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 862
2026-08-09T18:52:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1041 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (85 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:52:03 dist-test-slave-6zbq krb5kdc[1017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:52:03 dist-test-slave-6zbq krb5kdc[1017](info): set up 2 sockets
Aug 09 18:52:03 dist-test-slave-6zbq krb5kdc[1017](info): commencing operation
krb5kdc: starting...
W20260809 18:52:04.622047 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:52:04 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:52:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:52:04.729130 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36503
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:36503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:04.807488 1038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:04.807682 1038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:04.807703 1038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:04.808774 1038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:04.808806 1038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:04.808818 1038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:04.808830 1038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:04.808843 1038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:04.810195 1038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:36503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:04.810413 1038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:04.810583 1038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:04.814924 1044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:04.815061 1038 server_base.cc:1034] running on GCE node
W20260809 18:52:04.814934 1048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:04.814944 1045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:04.815382 1038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:04.815655 1038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:04.816800 1038 hybrid_clock.cc:648] HybridClock initialized: now 1786301524816786 us; error 32 us; skew 500 ppm
Aug 09 18:52:04 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301524, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:04.819037 1038 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:52:04.819437 1038 webserver.cc:460] Webserver started at http://127.23.218.254:33165/ using document root <none> and password file <none>
I20260809 18:52:04.819630 1038 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:04.819676 1038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:04.819797 1038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:04.820744 1038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "76c92be0dbba4f7ca7465442802e6d6d"
format_stamp: "Formatted at 2026-08-09 18:52:04 on dist-test-slave-6zbq"
server_key: "35e49a1d949d8b4d0e700c5951cc7b9b"
server_key_iv: "88cfb7a0b6a53e2a1469eb9296f8e070"
server_key_version: "encryptionkey@0"
I20260809 18:52:04.821055 1038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "76c92be0dbba4f7ca7465442802e6d6d"
format_stamp: "Formatted at 2026-08-09 18:52:04 on dist-test-slave-6zbq"
server_key: "35e49a1d949d8b4d0e700c5951cc7b9b"
server_key_iv: "88cfb7a0b6a53e2a1469eb9296f8e070"
server_key_version: "encryptionkey@0"
I20260809 18:52:04.822239 1038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:52:04.822916 1055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:04.823055 1038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:04.823138 1038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "76c92be0dbba4f7ca7465442802e6d6d"
format_stamp: "Formatted at 2026-08-09 18:52:04 on dist-test-slave-6zbq"
server_key: "35e49a1d949d8b4d0e700c5951cc7b9b"
server_key_iv: "88cfb7a0b6a53e2a1469eb9296f8e070"
server_key_version: "encryptionkey@0"
I20260809 18:52:04.823200 1038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:04.834281 1038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:04.835839 1038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:04.835953 1038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:04.839581 1038 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:36503
I20260809 18:52:04.839609 1130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:36503 every 8 connection(s)
I20260809 18:52:04.839895 1038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:04.840647 1131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:04.842789 1131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d: Bootstrap starting.
I20260809 18:52:04.843344 1131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:04.843358 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 1038
I20260809 18:52:04.843436 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:04.843593 24427 external_mini_cluster.cc:1428] Setting key 1fceb037beb7a167245a26737be651b1
I20260809 18:52:04.843616 1131 log.cc:824] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:04.844465 1131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d: No bootstrap required, opened a new log
I20260809 18:52:04.846772 1131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c92be0dbba4f7ca7465442802e6d6d" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36503 } }
I20260809 18:52:04.846930 1131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:04.846974 1131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c92be0dbba4f7ca7465442802e6d6d, State: Initialized, Role: FOLLOWER
I20260809 18:52:04.847100 1131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c92be0dbba4f7ca7465442802e6d6d" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36503 } }
I20260809 18:52:04.847170 1131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:04.847193 1131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:04.847237 1131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:04.847738 1131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c92be0dbba4f7ca7465442802e6d6d" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36503 } }
I20260809 18:52:04.847844 1131 leader_election.cc:302] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76c92be0dbba4f7ca7465442802e6d6d; no voters:
I20260809 18:52:04.847993 1131 leader_election.cc:288] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:04.848039 1137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:04.848192 1137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [term 1 LEADER]: Becoming Leader. State: Replica: 76c92be0dbba4f7ca7465442802e6d6d, State: Running, Role: LEADER
I20260809 18:52:04.848222 1131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:04.848340 1137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c92be0dbba4f7ca7465442802e6d6d" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36503 } }
I20260809 18:52:04.848778 1139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76c92be0dbba4f7ca7465442802e6d6d. Latest consensus state: current_term: 1 leader_uuid: "76c92be0dbba4f7ca7465442802e6d6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c92be0dbba4f7ca7465442802e6d6d" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36503 } } }
I20260809 18:52:04.848995 1139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:04.848925 1138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76c92be0dbba4f7ca7465442802e6d6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c92be0dbba4f7ca7465442802e6d6d" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36503 } } }
I20260809 18:52:04.849252 1138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:04.849560 1149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:04.850023 1149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 18:52:04 dist-test-slave-6zbq krb5kdc[1017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:04.852131 1149 catalog_manager.cc:1347] Generated new cluster ID: b3d8adccb1424b4e95734eece0665ad0
I20260809 18:52:04.852185 1149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:04.854013 1135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:04.844036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.23.218.254:36503)
0809 18:52:04.844194 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:04.844204 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:04.844235 (+ 31us) server_negotiation.cc:407] Connection header received
0809 18:52:04.844779 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:04.844788 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:04.844850 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:04.844997 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:04.845542 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:04.846925 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:04.847771 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:04.848378 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:04.848514 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:04.851082 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:04.851087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:04.851087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:04.851099 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:04.852715 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:04.853024 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:04.853027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:04.853028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:04.853115 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:04.853287 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:04.853287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:04.853288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:04.853506 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:04.853670 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:04.853735 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 18:52:04.853855 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260809 18:52:04.859758 1149 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:04.860222 1149 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:04.867133 1149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76c92be0dbba4f7ca7465442802e6d6d: Generated new TSK 0
I20260809 18:52:04.867286 1149 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:52:04.898437 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36503
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:04.992882 1165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:04.993055 1165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:04.993075 1165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:04.994196 1165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:04.994236 1165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:04.994263 1165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:04.995735 1165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:04.995957 1165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:04.996153 1165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 18:52:05.000845 1165 server_base.cc:1034] running on GCE node
W20260809 18:52:05.000830 1171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:05.000828 1174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:05.000816 1172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:05.001274 1165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:05.001505 1165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:05.002635 1165 hybrid_clock.cc:648] HybridClock initialized: now 1786301525002621 us; error 26 us; skew 500 ppm
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.004884 1165 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:52:05.005266 1165 webserver.cc:460] Webserver started at http://127.23.218.193:38909/ using document root <none> and password file <none>
I20260809 18:52:05.005470 1165 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:05.005514 1165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:05.005635 1165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:05.006618 1165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "198d8d0472834c2b89dfced23928b580"
format_stamp: "Formatted at 2026-08-09 18:52:05 on dist-test-slave-6zbq"
server_key: "7c76d66ac2ba32b0e9f1f16060facd4f"
server_key_iv: "d5c863eaf89011c08969410e7fdca242"
server_key_version: "encryptionkey@0"
I20260809 18:52:05.006920 1165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "198d8d0472834c2b89dfced23928b580"
format_stamp: "Formatted at 2026-08-09 18:52:05 on dist-test-slave-6zbq"
server_key: "7c76d66ac2ba32b0e9f1f16060facd4f"
server_key_iv: "d5c863eaf89011c08969410e7fdca242"
server_key_version: "encryptionkey@0"
I20260809 18:52:05.008085 1165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:05.008728 1184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.008888 1165 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:05.008970 1165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "198d8d0472834c2b89dfced23928b580"
format_stamp: "Formatted at 2026-08-09 18:52:05 on dist-test-slave-6zbq"
server_key: "7c76d66ac2ba32b0e9f1f16060facd4f"
server_key_iv: "d5c863eaf89011c08969410e7fdca242"
server_key_version: "encryptionkey@0"
I20260809 18:52:05.009027 1165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:05.017555 1165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:05.019111 1165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:05.019230 1165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:05.019415 1165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:05.019686 1165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:05.019719 1165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.019744 1165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:05.019761 1165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.025065 1165 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:35463
I20260809 18:52:05.025098 1364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:35463 every 8 connection(s)
I20260809 18:52:05.025358 1165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:05.033002 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 1165
I20260809 18:52:05.033098 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:52:05.033258 24427 external_mini_cluster.cc:1428] Setting key 565cfc40e890189ac3dbdb4a4ad0e765
I20260809 18:52:05.033502 1135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:05.025954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:40875 (local address 127.23.218.254:36503)
0809 18:52:05.025993 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:05.025995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:05.026158 (+ 163us) server_negotiation.cc:407] Connection header received
0809 18:52:05.027133 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:05.027135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:05.027146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:05.027203 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:05.027748 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.028222 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.029132 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.029329 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.029382 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:05.031191 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:05.031192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:05.031193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:05.031200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:05.032283 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.032619 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.032621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.032621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.032687 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.032889 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.032890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.032891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.032995 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:05.033049 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:05.033319 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 18:52:05.033409 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:52:05.033617 1373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:05.025930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36503 (local address 127.23.218.193:40875)
0809 18:52:05.026060 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:05.026091 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:05.027038 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:05.027268 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:05.027272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:05.027497 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:05.027662 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:05.027668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.028300 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.028301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:05.029054 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:05.029056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.029395 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.029396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:05.029517 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:05.029878 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:05.029886 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:05.031110 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:05.032353 (+ 1243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:05.032360 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:05.032362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:05.032555 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:05.032741 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:05.032742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:05.032742 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:05.032835 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:05.033049 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:05.033055 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:05.033202 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 18:52:05.033289 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 18:52:05.033425 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
I20260809 18:52:05.034125 1367 heartbeater.cc:347] Connected to a master server at 127.23.218.254:36503
I20260809 18:52:05.034209 1367 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:05.034391 1367 heartbeater.cc:511] Master 127.23.218.254:36503 requested a full tablet report, sending...
I20260809 18:52:05.034888 1076 ts_manager.cc:194] Registered new tserver with Master: 198d8d0472834c2b89dfced23928b580 (127.23.218.193:35463)
I20260809 18:52:05.035454 1076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:40875
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:52:05.054636 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36503
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:05.131853 1376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:05.132021 1376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:05.132042 1376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:05.133126 1376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:05.133163 1376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:05.133189 1376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:05.134605 1376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:05.134800 1376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:05.134987 1376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:05.139489 1390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:05.139519 1388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:05.139520 1376 server_base.cc:1034] running on GCE node
W20260809 18:52:05.139485 1394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:05.139918 1376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:05.140117 1376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:05.141235 1376 hybrid_clock.cc:648] HybridClock initialized: now 1786301525141226 us; error 27 us; skew 500 ppm
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.143491 1376 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:52:05.143896 1376 webserver.cc:460] Webserver started at http://127.23.218.194:41939/ using document root <none> and password file <none>
I20260809 18:52:05.144075 1376 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:05.144116 1376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:05.144234 1376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:05.145150 1376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "f5574fee1f684f9287fee505faffb008"
format_stamp: "Formatted at 2026-08-09 18:52:05 on dist-test-slave-6zbq"
server_key: "3efedf7750096e67ade1428a023705c9"
server_key_iv: "b27fa14c28a174d2cd11833972fb0bf7"
server_key_version: "encryptionkey@0"
I20260809 18:52:05.145452 1376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "f5574fee1f684f9287fee505faffb008"
format_stamp: "Formatted at 2026-08-09 18:52:05 on dist-test-slave-6zbq"
server_key: "3efedf7750096e67ade1428a023705c9"
server_key_iv: "b27fa14c28a174d2cd11833972fb0bf7"
server_key_version: "encryptionkey@0"
I20260809 18:52:05.146617 1376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:52:05.147264 1408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.147419 1376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:05.147500 1376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "f5574fee1f684f9287fee505faffb008"
format_stamp: "Formatted at 2026-08-09 18:52:05 on dist-test-slave-6zbq"
server_key: "3efedf7750096e67ade1428a023705c9"
server_key_iv: "b27fa14c28a174d2cd11833972fb0bf7"
server_key_version: "encryptionkey@0"
I20260809 18:52:05.147549 1376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:05.156101 1376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:05.157580 1376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:05.157685 1376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:05.157832 1376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:05.158039 1376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:05.158066 1376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.158083 1376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:05.158094 1376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.163300 1376 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:44501
I20260809 18:52:05.163336 1594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:44501 every 8 connection(s)
I20260809 18:52:05.163580 1376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:52:05.168589 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 1376
I20260809 18:52:05.168696 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:52:05.168870 24427 external_mini_cluster.cc:1428] Setting key 14d4f55d7a23444d87cb68a0281d2fe3
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:05.171952 1135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:05.164170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:33715 (local address 127.23.218.254:36503)
0809 18:52:05.164225 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:05.164227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:05.164366 (+ 139us) server_negotiation.cc:407] Connection header received
0809 18:52:05.165183 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:05.165184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:05.165196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:05.165246 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:05.165783 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.166262 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.167194 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.167383 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.167437 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:05.169434 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:05.169436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:05.169436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:05.169444 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:05.170725 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.171156 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.171158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.171158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.171212 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.171436 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.171437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.171437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.171524 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:05.171568 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:05.171826 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 18:52:05.171891 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:52:05.172156 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:05.164131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36503 (local address 127.23.218.194:33715)
0809 18:52:05.164262 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:05.164300 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:05.165089 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:05.165307 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:05.165311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:05.165530 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:05.165697 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:05.165700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.166363 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.166364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:05.167109 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:05.167111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.167448 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.167449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:05.167562 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:05.167922 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:05.167930 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:05.169352 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:05.170798 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:05.170805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:05.170807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:05.171073 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:05.171264 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:05.171265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:05.171265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:05.171383 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:05.171570 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:05.171575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:05.171712 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 18:52:05.171816 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 18:52:05.171966 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
I20260809 18:52:05.172606 1595 heartbeater.cc:347] Connected to a master server at 127.23.218.254:36503
I20260809 18:52:05.172693 1595 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:05.172850 1595 heartbeater.cc:511] Master 127.23.218.254:36503 requested a full tablet report, sending...
I20260809 18:52:05.173156 1076 ts_manager.cc:194] Registered new tserver with Master: f5574fee1f684f9287fee505faffb008 (127.23.218.194:44501)
I20260809 18:52:05.173518 1076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:33715
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:52:05.190546 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36503
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:05.267485 1603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:05.267666 1603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:05.267700 1603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:05.268790 1603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:05.268827 1603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:05.268860 1603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:05.270248 1603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:05.270476 1603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:05.270669 1603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 18:52:05.275197 1603 server_base.cc:1034] running on GCE node
W20260809 18:52:05.275175 1612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:05.275157 1610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:05.275169 1609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:05.275564 1603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:05.275763 1603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:05.276894 1603 hybrid_clock.cc:648] HybridClock initialized: now 1786301525276879 us; error 32 us; skew 500 ppm
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.279114 1603 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:52:05.279484 1603 webserver.cc:460] Webserver started at http://127.23.218.195:38513/ using document root <none> and password file <none>
I20260809 18:52:05.279660 1603 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:05.279709 1603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:05.279827 1603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:05.280735 1603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "4222c403a423467cafd645419664158e"
format_stamp: "Formatted at 2026-08-09 18:52:05 on dist-test-slave-6zbq"
server_key: "691b59f5329ab56f1050f7f1aae556e2"
server_key_iv: "d97ddc18f6b5100c34a07c50664f255b"
server_key_version: "encryptionkey@0"
I20260809 18:52:05.281038 1603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "4222c403a423467cafd645419664158e"
format_stamp: "Formatted at 2026-08-09 18:52:05 on dist-test-slave-6zbq"
server_key: "691b59f5329ab56f1050f7f1aae556e2"
server_key_iv: "d97ddc18f6b5100c34a07c50664f255b"
server_key_version: "encryptionkey@0"
I20260809 18:52:05.282197 1603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:52:05.282939 1625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.283108 1603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.283198 1603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "4222c403a423467cafd645419664158e"
format_stamp: "Formatted at 2026-08-09 18:52:05 on dist-test-slave-6zbq"
server_key: "691b59f5329ab56f1050f7f1aae556e2"
server_key_iv: "d97ddc18f6b5100c34a07c50664f255b"
server_key_version: "encryptionkey@0"
I20260809 18:52:05.283260 1603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:05.296875 1603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:05.298405 1603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:05.298527 1603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:05.298722 1603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:05.298976 1603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:05.299009 1603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.299033 1603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:05.299050 1603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:05.304636 1603 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:41363
I20260809 18:52:05.304699 1800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:41363 every 8 connection(s)
I20260809 18:52:05.304943 1603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:05.313113 1135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:05.305589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:54867 (local address 127.23.218.254:36503)
0809 18:52:05.305645 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:05.305647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:05.305801 (+ 154us) server_negotiation.cc:407] Connection header received
0809 18:52:05.306698 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:05.306699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:05.306708 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:05.306761 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:05.307281 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.307773 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.308729 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.308908 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.308950 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:05.310830 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:05.310832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:05.310832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:05.310839 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:05.312060 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.312378 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.312379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.312379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.312425 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.312627 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.312627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.312627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.312721 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:05.312777 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:05.312988 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 18:52:05.313051 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 18:52:05.313246 1803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:05.305543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36503 (local address 127.23.218.195:54867)
0809 18:52:05.305692 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:05.305730 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:05.306620 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:05.306809 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:05.306812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:05.307038 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:05.307212 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:05.307217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.307871 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.307873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:05.308663 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:05.308666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.308967 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.308968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:05.309066 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:05.309451 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:05.309459 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:05.310748 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:05.312119 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:05.312125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:05.312126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:05.312322 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:05.312468 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:05.312469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:05.312469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:05.312583 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:05.312779 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:05.312784 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:05.312889 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 18:52:05.312967 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 18:52:05.313078 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260809 18:52:05.313663 1801 heartbeater.cc:347] Connected to a master server at 127.23.218.254:36503
I20260809 18:52:05.313747 1801 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:05.313912 1801 heartbeater.cc:511] Master 127.23.218.254:36503 requested a full tablet report, sending...
I20260809 18:52:05.314239 1076 ts_manager.cc:194] Registered new tserver with Master: 4222c403a423467cafd645419664158e (127.23.218.195:41363)
I20260809 18:52:05.314320 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 1603
I20260809 18:52:05.314417 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:52:05.314592 24427 external_mini_cluster.cc:1428] Setting key 433173df18b09f453a7adddb80cf7cc8
I20260809 18:52:05.314718 1076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:54867
I20260809 18:52:05.315387 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/test-admin.keytab.
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.329417 24427 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 18:52:05.330842 1812 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:05.330141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.23.218.193:35463)
0809 18:52:05.330298 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:05.330302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:05.330310 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:05.330331 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:05.330332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:05.330363 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:05.330423 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:05.330766 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44914: BlockingRecv error: recv got EOF from 127.0.0.1:44914 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":63,"threads_started":1}
W20260809 18:52:05.330883 24427 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5cc' of type FILE
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.331738 24427 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.332561 24427 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:52:05.339082 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:05.332994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44930 (local address 127.23.218.193:35463)
0809 18:52:05.333030 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:05.333032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:05.333073 (+ 41us) server_negotiation.cc:407] Connection header received
0809 18:52:05.333112 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:05.333112 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:05.333121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:05.333169 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:05.333496 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.334185 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.334879 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.335185 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.335237 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:05.336950 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:05.336957 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:05.336957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:05.336963 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:05.338201 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.338539 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.338542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.338542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.338594 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.338720 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.338721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.338721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.338826 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:05.338925 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:05.338961 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 18:52:05.339017 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 18:52:05.340618 1812 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:05.340156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44942 (local address 127.23.218.193:35463)
0809 18:52:05.340207 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:05.340209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:05.340216 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:05.340285 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:05.340286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:05.340297 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:05.340340 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:05.340575 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44942: BlockingRecv error: recv got EOF from 127.0.0.1:44942 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 18:52:05.340677 24427 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5cc' of type FILE
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.341460 24427 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.342147 24427 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:52:05.347829 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:05.342613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44946 (local address 127.23.218.193:35463)
0809 18:52:05.342651 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:05.342653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:05.342685 (+ 32us) server_negotiation.cc:407] Connection header received
0809 18:52:05.342726 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:05.342726 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:05.342734 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:05.342783 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:05.343067 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.343494 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.344060 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.344248 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.344286 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:05.345935 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:05.345937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:05.345937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:05.345942 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:05.347088 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.347337 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.347337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.347337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.347379 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.347491 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.347491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.347491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.347558 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:05.347615 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:05.347698 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 18:52:05.347774 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 18:52:05.349143 1812 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:05.348670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44958 (local address 127.23.218.193:35463)
0809 18:52:05.348704 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:05.348706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:05.348777 (+ 71us) server_negotiation.cc:407] Connection header received
0809 18:52:05.348839 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:05.348840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:05.348848 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:05.348897 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:05.349100 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44958: BlockingRecv error: recv got EOF from 127.0.0.1:44958 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 18:52:05.349241 24427 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786301499543878-24427-0/krb5kdc/krb5cc' of type FILE
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.350124 24427 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:05.350899 24427 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:52:05.356442 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:05.351336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44964 (local address 127.23.218.193:35463)
0809 18:52:05.351370 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:05.351372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:05.351414 (+ 42us) server_negotiation.cc:407] Connection header received
0809 18:52:05.351477 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:05.351478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:05.351485 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:05.351528 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:05.351838 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.352251 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.352863 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:05.353039 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:05.353090 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:05.354793 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:05.354794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:05.354794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:05.354800 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:05.355737 (+ 937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.356005 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.356005 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.356005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.356043 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:05.356156 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:05.356156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:05.356156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:05.356224 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:05.356282 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:05.356342 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 18:52:05.356391 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:52:05.356926 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 1165
I20260809 18:52:05.373208 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 1376
I20260809 18:52:05.384769 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 1603
I20260809 18:52:05.411710 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 1038
2026-08-09T18:52:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1930 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1830](info): set up 2 sockets
Aug 09 18:52:05 dist-test-slave-6zbq krb5kdc[1830](info): commencing operation
krb5kdc: starting...
W20260809 18:52:06.542810 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:52:06 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:52:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.254.keytab.
Entry for principal oryx/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.254.keytab.
I20260809 18:52:06.648873 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38275
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.254.keytab
--principal=oryx/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:06.725502 1850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:06.725674 1850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:06.725694 1850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:06.726821 1850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:06.726861 1850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:06.726873 1850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:06.726886 1850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:06.726908 1850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:06.728315 1850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.254.keytab
--principal=oryx/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:06.728524 1850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:06.728715 1850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:06.733031 1856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:06.733062 1859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:06.733129 1850 server_base.cc:1034] running on GCE node
W20260809 18:52:06.733098 1857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:06.733385 1850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:06.733633 1850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:06.734766 1850 hybrid_clock.cc:648] HybridClock initialized: now 1786301526734751 us; error 28 us; skew 500 ppm
Aug 09 18:52:06 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301526, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:06.736943 1850 init.cc:377] Logged in from keytab as oryx/127.23.218.254@KRBTEST.COM (short username oryx)
I20260809 18:52:06.737294 1850 webserver.cc:460] Webserver started at http://127.23.218.254:45151/ using document root <none> and password file <none>
I20260809 18:52:06.737447 1850 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:06.737483 1850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:06.737562 1850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:06.738488 1850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "019912aa2162420e90731caaf172ec6b"
format_stamp: "Formatted at 2026-08-09 18:52:06 on dist-test-slave-6zbq"
server_key: "5125f7a39b5f4a512ae68af6e3728e74"
server_key_iv: "a4d75c56590b80ccb2463af04469791c"
server_key_version: "encryptionkey@0"
I20260809 18:52:06.738778 1850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "019912aa2162420e90731caaf172ec6b"
format_stamp: "Formatted at 2026-08-09 18:52:06 on dist-test-slave-6zbq"
server_key: "5125f7a39b5f4a512ae68af6e3728e74"
server_key_iv: "a4d75c56590b80ccb2463af04469791c"
server_key_version: "encryptionkey@0"
I20260809 18:52:06.739861 1850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:06.740449 1871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:06.740589 1850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:06.740667 1850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "019912aa2162420e90731caaf172ec6b"
format_stamp: "Formatted at 2026-08-09 18:52:06 on dist-test-slave-6zbq"
server_key: "5125f7a39b5f4a512ae68af6e3728e74"
server_key_iv: "a4d75c56590b80ccb2463af04469791c"
server_key_version: "encryptionkey@0"
I20260809 18:52:06.740718 1850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:06.759801 1850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:06.761387 1850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:06.761484 1850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:06.764945 1850 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:38275
I20260809 18:52:06.764979 1956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:38275 every 8 connection(s)
I20260809 18:52:06.765250 1850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:06.766064 1957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:06.768023 1957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: Bootstrap starting.
I20260809 18:52:06.768537 1957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:06.768782 1957 log.cc:824] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:06.769384 1957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: No bootstrap required, opened a new log
I20260809 18:52:06.770891 1957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } }
I20260809 18:52:06.771013 1957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:06.771044 1957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019912aa2162420e90731caaf172ec6b, State: Initialized, Role: FOLLOWER
I20260809 18:52:06.771175 1957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } }
I20260809 18:52:06.771236 1957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:06.771266 1957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:06.771312 1957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:06.772018 1957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } }
I20260809 18:52:06.772109 1957 leader_election.cc:302] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 019912aa2162420e90731caaf172ec6b; no voters:
I20260809 18:52:06.772254 1957 leader_election.cc:288] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:06.772289 1962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:06.772480 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 1850
I20260809 18:52:06.772490 1957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:06.772486 1962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 1 LEADER]: Becoming Leader. State: Replica: 019912aa2162420e90731caaf172ec6b, State: Running, Role: LEADER
I20260809 18:52:06.772576 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:06.772612 1962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } }
I20260809 18:52:06.772778 24427 external_mini_cluster.cc:1428] Setting key 7b0fdd89b175607b00cca0dcc958a45e
I20260809 18:52:06.772972 1967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 019912aa2162420e90731caaf172ec6b. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } }
I20260809 18:52:06.773046 1967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:06.773535 1977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:06.774082 1977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:06.773263 1965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } }
I20260809 18:52:06.775048 1965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:06.775533 1977 catalog_manager.cc:1347] Generated new cluster ID: 6cd7b2d124fd44a5817b7d4d2d228574
I20260809 18:52:06.775579 1977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:06.778775 1977 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:06.779443 1977 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 18:52:06 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:06.783738 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:06.773320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55064 (local address 127.23.218.254:38275)
0809 18:52:06.773493 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:06.773502 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:06.773590 (+ 88us) server_negotiation.cc:407] Connection header received
0809 18:52:06.773784 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:06.773791 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:06.773857 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:06.773991 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:06.774738 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:06.776165 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:06.777186 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:06.777792 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:06.777898 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:06.780487 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:06.780492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:06.780492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:06.780503 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:06.782247 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:06.782634 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:06.782637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:06.782637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:06.782732 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:06.782919 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:06.782920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:06.782920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:06.783166 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:06.783301 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:06.783399 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 18:52:06.783545 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260809 18:52:06.783962 1977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: Generated new TSK 0
I20260809 18:52:06.784104 1977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab.
Entry for principal oryx/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab.
I20260809 18:52:06.806459 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38275
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab
--principal=oryx/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:06.884274 2002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:06.884455 2002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:06.884485 2002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:06.885574 2002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:06.885613 2002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:06.885648 2002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:06.887063 2002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab
--principal=oryx/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:06.887274 2002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:06.887461 2002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:06.891948 2010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:06.891959 2009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:06.892112 2012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:06.892486 2002 server_base.cc:1034] running on GCE node
I20260809 18:52:06.892683 2002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:06.892889 2002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:06.894018 2002 hybrid_clock.cc:648] HybridClock initialized: now 1786301526894006 us; error 30 us; skew 500 ppm
Aug 09 18:52:06 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301526, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:06.896291 2002 init.cc:377] Logged in from keytab as oryx/127.23.218.193@KRBTEST.COM (short username oryx)
I20260809 18:52:06.896669 2002 webserver.cc:460] Webserver started at http://127.23.218.193:46793/ using document root <none> and password file <none>
I20260809 18:52:06.896847 2002 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:06.896893 2002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:06.897008 2002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:06.897938 2002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "d66a13491b2641e487b7554f63b90125"
format_stamp: "Formatted at 2026-08-09 18:52:06 on dist-test-slave-6zbq"
server_key: "9c89fe019c13c2adc9efc3fd7de83d42"
server_key_iv: "29dc686ef52acde441ef2f56cb8dd028"
server_key_version: "encryptionkey@0"
I20260809 18:52:06.898243 2002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "d66a13491b2641e487b7554f63b90125"
format_stamp: "Formatted at 2026-08-09 18:52:06 on dist-test-slave-6zbq"
server_key: "9c89fe019c13c2adc9efc3fd7de83d42"
server_key_iv: "29dc686ef52acde441ef2f56cb8dd028"
server_key_version: "encryptionkey@0"
I20260809 18:52:06.899407 2002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:52:06.900027 2020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:06.900169 2002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:06.900269 2002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "d66a13491b2641e487b7554f63b90125"
format_stamp: "Formatted at 2026-08-09 18:52:06 on dist-test-slave-6zbq"
server_key: "9c89fe019c13c2adc9efc3fd7de83d42"
server_key_iv: "29dc686ef52acde441ef2f56cb8dd028"
server_key_version: "encryptionkey@0"
I20260809 18:52:06.900324 2002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:06.906577 2002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:06.908056 2002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:06.908174 2002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:06.908361 2002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:06.908610 2002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:06.908641 2002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:06.908674 2002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:06.908699 2002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:06.913960 2002 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:41941
I20260809 18:52:06.913993 2193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:41941 every 8 connection(s)
I20260809 18:52:06.914245 2002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
Aug 09 18:52:06 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301526, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:06.920594 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 2002
I20260809 18:52:06.920691 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:52:06.920861 24427 external_mini_cluster.cc:1428] Setting key b6a3d42bb639e887e3c5e9d757c21768
I20260809 18:52:06.922127 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:06.914875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:42837 (local address 127.23.218.254:38275)
0809 18:52:06.914915 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:06.914917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:06.915097 (+ 180us) server_negotiation.cc:407] Connection header received
0809 18:52:06.915482 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:06.915484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:06.915493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:06.915538 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:06.916325 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:06.916801 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:06.917663 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:06.917849 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:06.917892 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:06.919697 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:06.919698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:06.919699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:06.919705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:06.920777 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:06.921236 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:06.921237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:06.921238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:06.921302 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:06.921537 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:06.921538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:06.921538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:06.921651 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:06.921715 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:06.921955 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 18:52:06.922038 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:52:06.922266 2198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:06.914865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.193:42837)
0809 18:52:06.914999 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:06.915029 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:06.915416 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:06.915585 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:06.915588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:06.916083 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:06.916258 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:06.916262 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:06.916863 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:06.916864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:06.917607 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:06.917608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:06.917901 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:06.917902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:06.917988 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:06.918364 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:06.918374 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:06.919617 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:06.920852 (+ 1235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:06.920859 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:06.920862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:06.921143 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:06.921363 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:06.921364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:06.921364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:06.921483 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:06.921712 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:06.921719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:06.921839 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 18:52:06.921923 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 18:52:06.922059 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260809 18:52:06.922694 2195 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38275
I20260809 18:52:06.922791 2195 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:06.922958 2195 heartbeater.cc:511] Master 127.23.218.254:38275 requested a full tablet report, sending...
I20260809 18:52:06.923492 1897 ts_manager.cc:194] Registered new tserver with Master: d66a13491b2641e487b7554f63b90125 (127.23.218.193:41941)
I20260809 18:52:06.924273 1897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:42837
WARNING: no policy specified for oryx/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab.
Entry for principal oryx/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab.
I20260809 18:52:06.942864 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38275
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab
--principal=oryx/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:07.021514 2202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:07.021703 2202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:07.021723 2202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:07.022855 2202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:07.022894 2202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:07.022921 2202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:07.024325 2202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab
--principal=oryx/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:07.024528 2202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:07.024719 2202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:07.029309 2214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.029286 2213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.029297 2218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:07.029507 2202 server_base.cc:1034] running on GCE node
I20260809 18:52:07.029670 2202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:07.029891 2202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:07.031041 2202 hybrid_clock.cc:648] HybridClock initialized: now 1786301527031024 us; error 27 us; skew 500 ppm
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:07.033272 2202 init.cc:377] Logged in from keytab as oryx/127.23.218.194@KRBTEST.COM (short username oryx)
I20260809 18:52:07.033682 2202 webserver.cc:460] Webserver started at http://127.23.218.194:36505/ using document root <none> and password file <none>
I20260809 18:52:07.033864 2202 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:07.033910 2202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:07.034027 2202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:07.035074 2202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "ea463fa92324405a9cad02790dd58f8f"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "7a2b9d26daba36c5098d6b8aa9455997"
server_key_iv: "1773078f63404c384e7286046a434a8b"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.035395 2202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "ea463fa92324405a9cad02790dd58f8f"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "7a2b9d26daba36c5098d6b8aa9455997"
server_key_iv: "1773078f63404c384e7286046a434a8b"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.036571 2202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:52:07.037245 2228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.037388 2202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:07.037467 2202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "ea463fa92324405a9cad02790dd58f8f"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "7a2b9d26daba36c5098d6b8aa9455997"
server_key_iv: "1773078f63404c384e7286046a434a8b"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.037534 2202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:07.048719 2202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:07.050606 2202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:07.050725 2202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:07.050907 2202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:07.051172 2202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:07.051204 2202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.051222 2202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:07.051245 2202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.056485 2202 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:44607
I20260809 18:52:07.056540 2412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:44607 every 8 connection(s)
I20260809 18:52:07.056787 2202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:07.064685 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.057392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:40661 (local address 127.23.218.254:38275)
0809 18:52:07.057435 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.057438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.057597 (+ 159us) server_negotiation.cc:407] Connection header received
0809 18:52:07.058042 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.058044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.058057 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.058106 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.058989 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.059484 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.060392 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.060587 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.060640 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.062469 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.062471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.062471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.062478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.063538 (+ 1060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.063868 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.063869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.063870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.063923 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.064120 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.064121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.064121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.064228 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.064284 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.064514 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.064588 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:52:07.064807 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.057335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.194:40661)
0809 18:52:07.057490 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.057526 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.057950 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.058179 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.058183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.058729 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.058897 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.058902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.059563 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.059564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.060312 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.060314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.060652 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.060653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.060766 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.061170 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.061179 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.062383 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.063606 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.063612 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.063614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.063806 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.063975 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.063976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.063976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.064066 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.064281 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.064287 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.064403 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.064485 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.064622 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260809 18:52:07.065233 2414 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38275
I20260809 18:52:07.065327 2414 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:07.065485 2414 heartbeater.cc:511] Master 127.23.218.254:38275 requested a full tablet report, sending...
I20260809 18:52:07.065826 1897 ts_manager.cc:194] Registered new tserver with Master: ea463fa92324405a9cad02790dd58f8f (127.23.218.194:44607)
I20260809 18:52:07.066140 1897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:40661
I20260809 18:52:07.066926 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 2202
I20260809 18:52:07.067018 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:52:07.067169 24427 external_mini_cluster.cc:1428] Setting key 5001b70cf0901cef23a741a0836f73bd
WARNING: no policy specified for oryx/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab.
Entry for principal oryx/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab.
I20260809 18:52:07.088757 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38275
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab
--principal=oryx/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:07.166671 2423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:07.166855 2423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:07.166882 2423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:07.167953 2423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:07.167986 2423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:07.168010 2423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:07.169368 2423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab
--principal=oryx/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:07.169536 2423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:07.169692 2423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:07.174687 2429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.174729 2430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.174729 2432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:07.174796 2423 server_base.cc:1034] running on GCE node
I20260809 18:52:07.175055 2423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:07.175254 2423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:07.176381 2423 hybrid_clock.cc:648] HybridClock initialized: now 1786301527176367 us; error 28 us; skew 500 ppm
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:07.178642 2423 init.cc:377] Logged in from keytab as oryx/127.23.218.195@KRBTEST.COM (short username oryx)
I20260809 18:52:07.179044 2423 webserver.cc:460] Webserver started at http://127.23.218.195:43329/ using document root <none> and password file <none>
I20260809 18:52:07.179224 2423 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:07.179270 2423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:07.179394 2423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:07.180296 2423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "01c2bc35b7ee44a8ab3386e497dae5e3"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "fbe1b28bec055a9d67216c1bf12d479d"
server_key_iv: "73a513d1ce7afb693b41b754eb06a854"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.180614 2423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "01c2bc35b7ee44a8ab3386e497dae5e3"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "fbe1b28bec055a9d67216c1bf12d479d"
server_key_iv: "73a513d1ce7afb693b41b754eb06a854"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.181784 2423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:07.182453 2446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.182590 2423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.182672 2423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "01c2bc35b7ee44a8ab3386e497dae5e3"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "fbe1b28bec055a9d67216c1bf12d479d"
server_key_iv: "73a513d1ce7afb693b41b754eb06a854"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.182737 2423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:07.197695 2423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:07.199616 2423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:07.199741 2423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:07.199949 2423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:07.200207 2423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:07.200239 2423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.200265 2423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:07.200282 2423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.205844 2423 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:35715
I20260809 18:52:07.205904 2625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:35715 every 8 connection(s)
I20260809 18:52:07.206151 2423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:07.212724 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 2423
I20260809 18:52:07.212831 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:52:07.213024 24427 external_mini_cluster.cc:1428] Setting key d1cb98a1c62f70b74d0b4631db076db7
I20260809 18:52:07.214097 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.206761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:59429 (local address 127.23.218.254:38275)
0809 18:52:07.206823 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.206826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.206985 (+ 159us) server_negotiation.cc:407] Connection header received
0809 18:52:07.207399 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.207400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.207410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.207458 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.208244 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.208736 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.209613 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.209801 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.209851 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.211668 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.211669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.211669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.211676 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.212692 (+ 1016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.213149 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.213150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.213151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.213218 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.213477 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.213478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.213479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.213619 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.213680 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.213969 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.214034 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 18:52:07.214214 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.206724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.195:59429)
0809 18:52:07.206884 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.206915 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.207322 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.207505 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.207508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.208008 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.208177 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.208181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.208796 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.208796 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.209539 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.209541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.209859 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.209860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.209945 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.210369 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.210377 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.211591 (+ 1214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.212763 (+ 1172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.212771 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.212773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.213045 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.213280 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.213281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.213281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.213419 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.213685 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.213691 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.213839 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.213943 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.214056 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260809 18:52:07.214653 2626 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38275
I20260809 18:52:07.214737 2626 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:07.214888 2626 heartbeater.cc:511] Master 127.23.218.254:38275 requested a full tablet report, sending...
I20260809 18:52:07.215221 1897 ts_manager.cc:194] Registered new tserver with Master: 01c2bc35b7ee44a8ab3386e497dae5e3 (127.23.218.195:35715)
I20260809 18:52:07.215565 1897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:59429
I20260809 18:52:07.224531 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.218.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.253.keytab.
Entry for principal oryx/127.23.218.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.253.keytab.
I20260809 18:52:07.246642 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.253:39781
--webserver_interface=127.23.218.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.253.keytab
--principal=oryx/127.23.218.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38275,127.23.218.253:39781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:07.325349 2635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:07.325528 2635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:07.325548 2635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:07.326676 2635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:07.326715 2635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:07.326728 2635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:07.326740 2635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:07.326764 2635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:07.328204 2635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38275,127.23.218.253:39781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.253.keytab
--principal=oryx/127.23.218.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.253:39781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.218.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
W20260809 18:52:07.328305 2635 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.23.218.254:38275,127.23.218.253:39781'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 18:52:07.328441 2635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:07.328665 2635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 18:52:07.333091 2635 server_base.cc:1034] running on GCE node
W20260809 18:52:07.333074 2649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.333069 2645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.333062 2644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:07.333539 2635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:07.333812 2635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:07.334950 2635 hybrid_clock.cc:648] HybridClock initialized: now 1786301527334933 us; error 29 us; skew 500 ppm
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:07.337267 2635 init.cc:377] Logged in from keytab as oryx/127.23.218.253@KRBTEST.COM (short username oryx)
I20260809 18:52:07.337647 2635 webserver.cc:460] Webserver started at http://127.23.218.253:43143/ using document root <none> and password file <none>
I20260809 18:52:07.337860 2635 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:07.337908 2635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:07.338032 2635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:07.339066 2635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data/instance:
uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "5dc4be1a805f00c00198804f4efbd937"
server_key_iv: "49bac313540eb6142156ab83ef00b549"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.339380 2635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal/instance:
uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "5dc4be1a805f00c00198804f4efbd937"
server_key_iv: "49bac313540eb6142156ab83ef00b549"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.340627 2635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:07.341310 2659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.341459 2635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:07.341540 2635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal
uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "5dc4be1a805f00c00198804f4efbd937"
server_key_iv: "49bac313540eb6142156ab83ef00b549"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.341598 2635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:07.352223 2635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:07.354126 2635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:07.354254 2635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:07.357790 2635 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.253:39781
I20260809 18:52:07.357821 2738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.253:39781 every 8 connection(s)
I20260809 18:52:07.358112 2635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
I20260809 18:52:07.358997 2739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:07.360386 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 2635
I20260809 18:52:07.360399 2739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.218.253@KRBTEST.COM
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.253@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:07.371619 2742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.360928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41038 (local address 127.23.218.253:39781)
0809 18:52:07.361127 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.361137 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.361177 (+ 40us) server_negotiation.cc:407] Connection header received
0809 18:52:07.361634 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.361643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.361668 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.361760 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.362311 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.363525 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.364477 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.365025 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.365125 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.367790 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.367799 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.367800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.367807 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.370046 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.370425 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.370428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.370428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.370514 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.370754 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.370754 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.370755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.371014 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.371130 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.371301 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.371406 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260809 18:52:07.373912 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.361107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.23.218.254:38275)
0809 18:52:07.361155 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.361157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.361338 (+ 181us) server_negotiation.cc:407] Connection header received
0809 18:52:07.361751 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.361753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.361764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.361827 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.362374 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.363077 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.366264 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.366579 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.366626 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.368939 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.368940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.368941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.368949 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.372576 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.372979 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.372980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.372981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.373038 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.373235 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.373237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.373237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.373437 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.373496 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.373744 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.373823 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:52:07.374217 2745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.361027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55066)
0809 18:52:07.361213 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.361250 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.361650 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.361857 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.361859 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.362098 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.362253 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.362257 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.363193 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.363194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.366154 (+ 2960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.366158 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.366719 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.366720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.366796 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.367289 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.367350 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.368836 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.372652 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.372660 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.372662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.372917 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.373102 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.373103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.373103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.373165 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.373537 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.373545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.373597 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.373997 (+ 400us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.374110 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":83,"threads_started":1}
I20260809 18:52:07.374720 2739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.253@KRBTEST.COM for oryx/127.23.218.253@KRBTEST.COM
I20260809 18:52:07.379698 2745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.374916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.0.0.1:41042)
0809 18:52:07.374955 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.374964 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.374998 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.375089 (+ 91us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.375089 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.375191 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.375218 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.375219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.375729 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.375730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.376170 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.376171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.376435 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.376436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.376477 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.376727 (+ 250us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.376730 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.377716 (+ 986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.378876 (+ 1160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.378877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.378878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.379023 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.379172 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.379173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.379174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.379224 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.379414 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.379415 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.379450 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.379509 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.379599 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 18:52:07.379750 2742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.374954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.23.218.253:39781)
0809 18:52:07.374985 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.374987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.374994 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:07.375037 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.375037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.375044 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.375087 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.375260 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.375681 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.376217 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.376393 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.376433 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.377787 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.377788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.377788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.377794 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.378798 (+ 1004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.379078 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.379079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.379079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.379122 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.379272 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.379273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.379273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.379363 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.379414 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.379536 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.379645 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":14}
I20260809 18:52:07.380802 2739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Bootstrap starting.
I20260809 18:52:07.381572 2739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:07.381821 2739 log.cc:824] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:07.382481 2739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: No bootstrap required, opened a new log
I20260809 18:52:07.383769 2739 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } }
I20260809 18:52:07.383898 2739 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:07.383932 2739 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fa72c6ae4e4a18b02bb7ba0dd3deec, State: Initialized, Role: FOLLOWER
I20260809 18:52:07.384047 2739 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } }
I20260809 18:52:07.384255 2749 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } } }
I20260809 18:52:07.384310 2739 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:07.384325 2749 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: This master's current role is: FOLLOWER
W20260809 18:52:07.385756 2769 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:52:07.385933 2769 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:52:07.387216 2635 master_runner.cc:428] Detected that this master f4fa72c6ae4e4a18b02bb7ba0dd3deec is joining an existing cluster
I20260809 18:52:07.387257 2635 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.218.253:39781
I20260809 18:52:07.394075 2742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.388309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.23.218.253:39781)
0809 18:52:07.388344 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.388346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.388593 (+ 247us) server_negotiation.cc:407] Connection header received
0809 18:52:07.388625 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.388626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.388633 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.388672 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.389013 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.389685 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.390597 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.390851 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.390912 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.391697 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.391698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.391698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.391704 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.393013 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.393435 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.393435 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.393436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.393482 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.393634 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.393635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.393635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.393737 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.393768 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.393923 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.393990 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:52:07.394164 2777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.388087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55080)
0809 18:52:07.388247 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.388257 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.388332 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.388496 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.388496 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.388685 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.388773 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.388775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.389438 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.389439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.390182 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.390184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.390671 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.390672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.390727 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.391120 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.391124 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.391531 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.393140 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.393141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.393142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.393393 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.393578 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.393579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.393579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.393628 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.393868 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.393869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.393905 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.393970 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.394045 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260809 18:52:07.394378 2779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.388283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.0.0.1:41058)
0809 18:52:07.388536 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.388544 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.388586 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.388674 (+ 88us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.388675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.388890 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.388942 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.388943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.389777 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.389778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.390530 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.390532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.390912 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.390913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.390986 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.391254 (+ 268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.391258 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.391632 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.393085 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.393086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.393087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.393360 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.393535 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.393535 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.393535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.393577 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.393794 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.393795 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.393833 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.394178 (+ 345us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.394256 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":110,"threads_started":1}
I20260809 18:52:07.394779 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.388167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.23.218.254:38275)
0809 18:52:07.388207 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.388209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.388308 (+ 99us) server_negotiation.cc:407] Connection header received
0809 18:52:07.388412 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.388413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.388424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.388478 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.388858 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.389371 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.390265 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.390584 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.390668 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.391613 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.391615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.391615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.391622 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.393066 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.393455 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.393456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.393456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.393517 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.393686 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.393687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.393688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.393806 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.393868 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.394002 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.394071 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301526, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.254@KRBTEST.COM for oryx/127.23.218.253@KRBTEST.COM
I20260809 18:52:07.404506 2742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.396866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.23.218.253:39781)
0809 18:52:07.396903 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.396905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.396968 (+ 63us) server_negotiation.cc:407] Connection header received
0809 18:52:07.397037 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.397038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.397046 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.397108 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.397429 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.398107 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.399098 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.399354 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.399391 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.402281 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.402282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.402282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.402289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.403472 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.403808 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.403809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.403809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.403861 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.404020 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.404021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.404021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.404137 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.404190 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.404355 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.404424 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:52:07.404528 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.396779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.0.0.1:41068)
0809 18:52:07.396886 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.396922 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.396985 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.397112 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.397114 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.397330 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.397370 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.397372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.398168 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.398169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.399002 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.399005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.399460 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.399461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.399596 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.400799 (+ 1203us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.400804 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.402200 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.403546 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.403553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.403555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.403744 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.403914 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.403915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.403915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.403971 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.404195 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.404202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.404251 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.404321 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.404427 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":63,"threads_started":1}
I20260809 18:52:07.405340 1897 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.218.253:39781: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec"
member_type: NON_VOTER
last_known_addr {
host: "127.23.218.253"
port: 39781
}
attrs {
promote: true
}
}
dest_uuid: "019912aa2162420e90731caaf172ec6b"
cas_config_opid_index: -1
I20260809 18:52:07.405637 1897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } }
I20260809 18:52:07.406101 2786 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f4fa72c6ae4e4a18b02bb7ba0dd3deec (127.23.218.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } } }
I20260809 18:52:07.406385 2697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:07.407044 2697 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Refusing update from remote peer 019912aa2162420e90731caaf172ec6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 18:52:07.407071 2786 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.218.253:39781
I20260809 18:52:07.407143 2786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } } }
I20260809 18:52:07.407205 2786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:07.407398 2635 master.cc:537] Master@127.23.218.253:39781 shutting down...
I20260809 18:52:07.407390 2784 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 18:52:07.408030 1877 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 18:52:07.408433 1877 consensus_peers.cc:487] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b -> Peer f4fa72c6ae4e4a18b02bb7ba0dd3deec (127.23.218.253:39781): Couldn't send request to peer f4fa72c6ae4e4a18b02bb7ba0dd3deec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 18:52:07.409921 2635 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:07.409996 2635 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:07.410045 2635 tablet_replica.cc:330] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: stopping tablet replica
I20260809 18:52:07.423120 2635 master.cc:552] Master@127.23.218.253:39781 shutdown complete.
I20260809 18:52:07.423157 2635 master_runner.cc:305] Clearing existing system tablet
I20260809 18:52:07.423264 2635 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:07.423303 2635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:07.423832 2635 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:07.424237 2791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.424347 2635 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:07.424409 2635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.424434 2635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal
uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "5dc4be1a805f00c00198804f4efbd937"
server_key_iv: "49bac313540eb6142156ab83ef00b549"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.425208 2635 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:07.426508 2635 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 18:52:07.426576 2635 log.cc:1196] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 18:52:07.426769 2635 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Deleting consensus metadata
I20260809 18:52:07.426882 2635 master_runner.cc:315] Copying system tablet from 127.23.218.254:38275
I20260809 18:52:07.427430 2635 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: tablet copy: Beginning tablet copy session from remote peer at address 127.23.218.254:38275
I20260809 18:52:07.432806 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.427742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55092 (local address 127.23.218.254:38275)
0809 18:52:07.427781 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.427783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.427825 (+ 42us) server_negotiation.cc:407] Connection header received
0809 18:52:07.427891 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.427891 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.427899 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.427942 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.428236 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.428679 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.429659 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.429833 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.429873 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.430728 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.430729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.430730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.430735 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.431901 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.432250 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.432251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.432252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.432292 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.432443 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.432443 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.432443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.432524 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.432565 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.432702 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.432752 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 18:52:07.432885 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.427658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55092)
0809 18:52:07.427768 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.427777 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.427835 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.427947 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.427948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.428118 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.428180 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.428182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.428743 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.428744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.429590 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.429593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.429889 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.429890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.429953 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.430285 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.430290 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.430647 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.431984 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.431985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.431985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.432188 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.432335 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.432335 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.432336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.432393 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.432573 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.432574 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.432615 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.432688 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.432779 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":72,"threads_started":1}
I20260809 18:52:07.433190 1929 tablet_copy_service.cc:140] P 019912aa2162420e90731caaf172ec6b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f4fa72c6ae4e4a18b02bb7ba0dd3deec ({username='oryx', principal='oryx/127.23.218.253@KRBTEST.COM'} at 127.0.0.1:55092)
I20260809 18:52:07.433261 1929 tablet_copy_service.cc:161] P 019912aa2162420e90731caaf172ec6b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f4fa72c6ae4e4a18b02bb7ba0dd3deec at {username='oryx', principal='oryx/127.23.218.253@KRBTEST.COM'} at 127.0.0.1:55092: session id = f4fa72c6ae4e4a18b02bb7ba0dd3deec-00000000000000000000000000000000
I20260809 18:52:07.433789 1929 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 18:52:07.434377 2635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 18:52:07.435570 2635 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: tablet copy: Starting download of 0 data blocks...
I20260809 18:52:07.435717 2635 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: tablet copy: Starting download of 1 WAL segments...
I20260809 18:52:07.436503 2635 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 18:52:07.437335 1929 tablet_copy_service.cc:342] P 019912aa2162420e90731caaf172ec6b: Request end of tablet copy session f4fa72c6ae4e4a18b02bb7ba0dd3deec-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.218.253@KRBTEST.COM'} at 127.0.0.1:55092
I20260809 18:52:07.437389 1929 tablet_copy_service.cc:433] P 019912aa2162420e90731caaf172ec6b: ending tablet copy session f4fa72c6ae4e4a18b02bb7ba0dd3deec-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f4fa72c6ae4e4a18b02bb7ba0dd3deec
I20260809 18:52:07.439463 2635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:07.440433 2806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.440527 2809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:07.440658 2635 server_base.cc:1034] running on GCE node
W20260809 18:52:07.440713 2802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:07.440842 2635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:07.440966 2635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:07.442075 2635 hybrid_clock.cc:648] HybridClock initialized: now 1786301527442060 us; error 30 us; skew 500 ppm
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:07.443217 2635 init.cc:377] Logged in from keytab as oryx/127.23.218.253@KRBTEST.COM (short username oryx)
I20260809 18:52:07.443454 2635 webserver.cc:460] Webserver started at http://127.23.218.253:33251/ using document root <none> and password file <none>
I20260809 18:52:07.443562 2635 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:07.443594 2635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:07.444110 2635 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:07.444597 2825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.444713 2635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.444756 2635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal
uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "5dc4be1a805f00c00198804f4efbd937"
server_key_iv: "49bac313540eb6142156ab83ef00b549"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.444802 2635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:07.458204 2635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:07.459566 2635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:07.462419 2635 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.253:39781
I20260809 18:52:07.462600 2922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.253:39781 every 8 connection(s)
I20260809 18:52:07.462733 2635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
I20260809 18:52:07.463574 2923 sys_catalog.cc:263] Verifying existing consensus state
I20260809 18:52:07.464093 2923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Bootstrap starting.
I20260809 18:52:07.466910 2923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:52:07.467142 2923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Bootstrap complete.
I20260809 18:52:07.467338 2923 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } }
I20260809 18:52:07.467423 2923 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f4fa72c6ae4e4a18b02bb7ba0dd3deec, State: Initialized, Role: LEARNER
I20260809 18:52:07.467468 2923 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } }
I20260809 18:52:07.467643 2923 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:07.467640 2926 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } } }
I20260809 18:52:07.467765 2926 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: This master's current role is: LEARNER
I20260809 18:52:07.469193 2945 catalog_manager.cc:1259] Loaded cluster ID: 6cd7b2d124fd44a5817b7d4d2d228574
I20260809 18:52:07.469242 2945 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: loading cluster ID for follower catalog manager: success
I20260809 18:52:07.470132 2945 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: acquiring CA information for follower catalog manager: success
I20260809 18:52:07.470427 2945 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 18:52:07.478825 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.473072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.23.218.253:39781)
0809 18:52:07.473201 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.473203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.473211 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:07.473230 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.473230 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.473239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.473297 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.473792 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.474556 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.475224 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.475437 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.475490 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.476531 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.476533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.476533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.476540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.477791 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.478086 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.478087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.478088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.478144 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.478304 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.478305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.478306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.478453 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.478538 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.478660 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.478731 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for oryx/127.23.218.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.252.keytab.
Entry for principal oryx/127.23.218.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.252.keytab.
I20260809 18:52:07.504545 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.252:43931
--webserver_interface=127.23.218.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.252.keytab
--principal=oryx/127.23.218.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38275,127.23.218.253:39781,127.23.218.252:43931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:07.583801 2958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:07.583973 2958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:07.583993 2958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:07.585111 2958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:07.585151 2958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:07.585171 2958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:07.585182 2958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:07.585192 2958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:07.586647 2958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38275,127.23.218.253:39781,127.23.218.252:43931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.252.keytab
--principal=oryx/127.23.218.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.252:43931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.218.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:07.586851 2958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:07.587050 2958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:07.591290 2967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.591324 2969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.591321 2966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:07.591460 2958 server_base.cc:1034] running on GCE node
I20260809 18:52:07.591663 2958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:07.591934 2958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:07.593070 2958 hybrid_clock.cc:648] HybridClock initialized: now 1786301527593054 us; error 29 us; skew 500 ppm
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:07.595300 2958 init.cc:377] Logged in from keytab as oryx/127.23.218.252@KRBTEST.COM (short username oryx)
I20260809 18:52:07.595667 2958 webserver.cc:460] Webserver started at http://127.23.218.252:39639/ using document root <none> and password file <none>
I20260809 18:52:07.595844 2958 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:07.595888 2958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:07.596004 2958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:07.597002 2958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data/instance:
uuid: "1e4676ef914b45dd963ce1983aa14032"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "c3e2eab570c6a0325a7f44dc3873c82f"
server_key_iv: "e6b016d16c589f7dfa07b95a0db689d0"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.597323 2958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal/instance:
uuid: "1e4676ef914b45dd963ce1983aa14032"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "c3e2eab570c6a0325a7f44dc3873c82f"
server_key_iv: "e6b016d16c589f7dfa07b95a0db689d0"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.598516 2958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:52:07.599131 2978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.599280 2958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:07.599360 2958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal
uuid: "1e4676ef914b45dd963ce1983aa14032"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "c3e2eab570c6a0325a7f44dc3873c82f"
server_key_iv: "e6b016d16c589f7dfa07b95a0db689d0"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.599423 2958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:07.611765 2958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:07.613248 2958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:07.613369 2958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:07.616916 2958 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.252:43931
I20260809 18:52:07.616954 3051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.252:43931 every 8 connection(s)
I20260809 18:52:07.617231 2958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
I20260809 18:52:07.618079 3052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:07.618737 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 2958
I20260809 18:52:07.619843 3052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.252@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.218.252@KRBTEST.COM
I20260809 18:52:07.627949 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.620505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55108 (local address 127.23.218.254:38275)
0809 18:52:07.620549 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.620550 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.620628 (+ 78us) server_negotiation.cc:407] Connection header received
0809 18:52:07.620692 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.620693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.620705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.620749 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.621348 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.622116 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.622991 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.623215 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.623280 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.625194 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.625196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.625196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.625204 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.626700 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.627079 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.627081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.627081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.627142 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.627352 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.627354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.627354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.627468 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.627527 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.627769 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.627859 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:52:07.628064 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.620376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55108)
0809 18:52:07.620531 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.620570 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.620634 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.620772 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.620775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.621092 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.621216 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.621223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.622195 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.622196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.622910 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.622913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.623316 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.623317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.623370 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.623800 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.623809 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.625102 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.626775 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.626781 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.626782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.627011 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.627201 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.627202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.627202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.627285 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.627524 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.627531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.627653 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.627735 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.627871 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260809 18:52:07.628378 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.619273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35676 (local address 127.23.218.252:43931)
0809 18:52:07.619429 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.619436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.619458 (+ 22us) server_negotiation.cc:407] Connection header received
0809 18:52:07.619829 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.619835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.619877 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.619993 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.620673 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.621912 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.622787 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.623141 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.623219 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.625589 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.625596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.625597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.625603 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.627211 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.627491 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.627494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.627495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.627548 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.627712 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.627713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.627713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.627837 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.627960 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.628218 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.628291 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260809 18:52:07.628576 3052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.252@KRBTEST.COM for oryx/127.23.218.253@KRBTEST.COM
I20260809 18:52:07.633704 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.628784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.0.0.1:41108)
0809 18:52:07.628821 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.628830 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.628863 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.629035 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.629036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.629152 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.629191 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.629193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.629780 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.629780 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.630235 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.630236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.630573 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.630574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.630619 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.630889 (+ 270us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.630893 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.631874 (+ 981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.633044 (+ 1170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.633045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.633045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.633198 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.633335 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.633336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.633336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.633374 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.633521 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.633522 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.633548 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.633595 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.633652 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 18:52:07.633726 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.628848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.23.218.253:39781)
0809 18:52:07.628886 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.628887 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.628895 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:07.628921 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.628922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.628933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.628988 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.629262 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.629724 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.630290 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.630523 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.630569 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.631952 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.631954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.631954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.631961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.632976 (+ 1015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.633248 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.633248 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.633249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.633291 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.633405 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.633405 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.633406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.633490 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.633529 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.633605 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.633667 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:52:07.634086 3052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.252@KRBTEST.COM for oryx/127.23.218.252@KRBTEST.COM
I20260809 18:52:07.639065 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.634279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.0.0.1:35684)
0809 18:52:07.634315 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.634324 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.634375 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.634479 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.634479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.634576 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.634607 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.634608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.635113 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.635113 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.635602 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.635603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.635903 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.635904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.635935 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.636205 (+ 270us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.636208 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.637133 (+ 925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.638423 (+ 1290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.638423 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.638424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.638559 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.638693 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.638694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.638694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.638734 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.638882 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.638882 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.638910 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.638952 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.639013 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 18:52:07.639135 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.634305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.23.218.252:43931)
0809 18:52:07.634375 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.634377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.634385 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:07.634422 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.634423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.634431 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.634470 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.634649 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.635062 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.635661 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.635852 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.635901 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.637199 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.637200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.637201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.637206 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.638364 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.638611 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.638612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.638612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.638655 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.638767 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.638767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.638768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.638845 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.638885 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.638968 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.639053 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":27}
I20260809 18:52:07.640172 3052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: Bootstrap starting.
I20260809 18:52:07.640877 3052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:07.641101 3052 log.cc:824] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:07.641655 3052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: No bootstrap required, opened a new log
I20260809 18:52:07.642997 3052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } }
I20260809 18:52:07.643126 3052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:07.643147 3052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4676ef914b45dd963ce1983aa14032, State: Initialized, Role: FOLLOWER
I20260809 18:52:07.643227 3052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } }
I20260809 18:52:07.643471 3052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:07.643443 3057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } } }
I20260809 18:52:07.643528 3057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 18:52:07.645164 3070 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:52:07.645218 3070 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:52:07.647205 2958 master_runner.cc:428] Detected that this master 1e4676ef914b45dd963ce1983aa14032 is joining an existing cluster
I20260809 18:52:07.647243 2958 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.218.252:43931
I20260809 18:52:07.653947 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.648406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55120 (local address 127.23.218.254:38275)
0809 18:52:07.648441 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.648443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.648450 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:07.648469 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.648469 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.648477 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.648521 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.648927 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.649655 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.650729 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.651010 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.651044 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.651908 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.651909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.651909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.651916 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.653028 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.653312 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.653313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.653313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.653363 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.653516 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.653516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.653516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.653617 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.653670 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.653793 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.653860 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:52:07.653964 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.648070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55120)
0809 18:52:07.648298 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.648308 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.648353 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.648563 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.648563 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.648774 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.648845 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.648846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.649740 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.649741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.650643 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.650644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.651114 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.651115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.651166 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.651524 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.651528 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.651820 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.653096 (+ 1276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.653096 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.653097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.653253 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.653413 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.653413 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.653414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.653467 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.653668 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.653669 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.653707 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.653766 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.653861 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":22,"thread_start_us":80,"threads_started":1}
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301526, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.254@KRBTEST.COM for oryx/127.23.218.252@KRBTEST.COM
I20260809 18:52:07.673287 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.655711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.0.0.1:35692)
0809 18:52:07.655757 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.655768 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.655808 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.656114 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.656115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.656272 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.656322 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.656323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.656888 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.656889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.657689 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.657691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.657990 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.657991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.658054 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.658382 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.658386 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.659518 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.672207 (+ 12689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.672208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.672208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.672405 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.672655 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.672656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.672656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.672721 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.673005 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.673006 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.673051 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.673099 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.673185 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 18:52:07.673712 3088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.655876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.23.218.252:43931)
0809 18:52:07.656002 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.656003 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.656009 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:52:07.656035 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.656036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.656042 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.656104 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.656384 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.656828 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.657755 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.657934 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.657976 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.659609 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.659611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.659611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.659617 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.672121 (+ 12504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.672529 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.672530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.672530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.672583 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.672781 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.672782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.672782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.672892 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.673447 (+ 555us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.673486 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.673612 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260809 18:52:07.674121 1897 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.218.252:43931: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1e4676ef914b45dd963ce1983aa14032"
member_type: NON_VOTER
last_known_addr {
host: "127.23.218.252"
port: 43931
}
attrs {
promote: true
}
}
dest_uuid: "019912aa2162420e90731caaf172ec6b"
cas_config_opid_index: 5
I20260809 18:52:07.674389 1897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } }
I20260809 18:52:07.674635 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.648138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.23.218.253:39781)
0809 18:52:07.648179 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.648181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.648477 (+ 296us) server_negotiation.cc:407] Connection header received
0809 18:52:07.648571 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.648572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.648582 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.648609 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.649068 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.649777 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.650841 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.651131 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.651171 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.671569 (+ 20398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.671570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.671570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.671577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.673463 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.673836 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.673838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.673838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.673890 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.674084 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.674085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.674085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.674203 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.674244 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.674470 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.674544 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:52:07.674952 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.648154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.0.0.1:41116)
0809 18:52:07.648413 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.648423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.648503 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.648670 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.648671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.648911 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.648978 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.648980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.649873 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.649873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.650742 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.650745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.651241 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.651242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.651304 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.671011 (+ 19707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.671015 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.671475 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.673559 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.673560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.673561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.673744 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.673953 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.673954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.673954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.674010 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.674295 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.674296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.674334 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.674734 (+ 400us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.674833 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":5,"thread_start_us":76,"threads_started":1}
I20260809 18:52:07.674881 2786 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1e4676ef914b45dd963ce1983aa14032 (127.23.218.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.675581 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.648285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.0.0.1:35688)
0809 18:52:07.648541 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.648549 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.648602 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.648731 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.648731 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.648917 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.648978 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.648980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.649760 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.649761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.650640 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.650642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.651109 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.651110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.651180 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.670753 (+ 19573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.670757 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.671266 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.674164 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.674165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.674166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.674409 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.674874 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.674875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.674875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.674954 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.675296 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.675297 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.675340 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.675401 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.675480 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":31,"thread_start_us":73,"threads_started":1}
I20260809 18:52:07.675779 2786 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.218.252:43931
I20260809 18:52:07.675843 3054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.648474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.23.218.252:43931)
0809 18:52:07.648500 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.648501 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.648586 (+ 85us) server_negotiation.cc:407] Connection header received
0809 18:52:07.648657 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.648658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.648665 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.648687 (+ 22us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.649041 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.649693 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.650726 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.650992 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.651025 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.671349 (+ 20324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.671349 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.671350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.671355 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.673472 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.674600 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.674601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.674602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.674654 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.675120 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.675120 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.675121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.675229 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.675288 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.675714 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.675776 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 18:52:07.675850 2786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.675918 2786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:07.676193 2958 master.cc:537] Master@127.23.218.252:43931 shutting down...
W20260809 18:52:07.676817 1874 consensus_peers.cc:487] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b -> Peer 1e4676ef914b45dd963ce1983aa14032 (127.23.218.252:43931): Couldn't send request to peer 1e4676ef914b45dd963ce1983aa14032. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 18:52:07.678977 2958 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:07.679068 2958 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:07.679112 2958 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: stopping tablet replica
I20260809 18:52:07.679592 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.676325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.0.0.1:41126)
0809 18:52:07.676544 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.676552 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.676590 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.676830 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.676831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.676833 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:07.676867 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.676869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.677649 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.677650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.678618 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.678621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.679337 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.679338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.679408 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.679437 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.679494 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.679527 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260809 18:52:07.679942 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.676400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.23.218.253:39781)
0809 18:52:07.676432 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.676434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.676635 (+ 201us) server_negotiation.cc:407] Connection header received
0809 18:52:07.676654 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.676655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.676676 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.676728 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:07.676924 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.677549 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.678693 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.679252 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.679325 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.679400 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.679846 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.679883 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 18:52:07.680153 1827 connection.cc:535] client connection to 127.23.218.252:43931 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 18:52:07.680249 2873 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 LEARNER]: Refusing update from remote peer 019912aa2162420e90731caaf172ec6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 18:52:07.680444 3089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 18:52:07.681296 2873 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1e4676ef914b45dd963ce1983aa14032 (127.23.218.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.682466 3091 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.682564 3091 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: This master's current role is: LEARNER
I20260809 18:52:07.682569 2926 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 019912aa2162420e90731caaf172ec6b. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: NON_VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: true } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.682772 3089 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: attempting to promote NON_VOTER f4fa72c6ae4e4a18b02bb7ba0dd3deec to VOTER
I20260809 18:52:07.682792 2926 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: This master's current role is: LEARNER
I20260809 18:52:07.682922 3089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } }
I20260809 18:52:07.683319 2873 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 LEARNER]: Refusing update from remote peer 019912aa2162420e90731caaf172ec6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 18:52:07.683354 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.683239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.0.0.1:35708)
0809 18:52:07.683282 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.683328 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 18:52:07.683495 3089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 18:52:07.683624 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.683540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.0.0.1:35710)
0809 18:52:07.683581 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.683605 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 18:52:07.683701 1874 consensus_peers.cc:487] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b -> Peer 1e4676ef914b45dd963ce1983aa14032 (127.23.218.252:43931): Couldn't send request to peer 1e4676ef914b45dd963ce1983aa14032. Status: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43931: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 18:52:07.684065 2784 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f4fa72c6ae4e4a18b02bb7ba0dd3deec (127.23.218.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.684211 2873 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f4fa72c6ae4e4a18b02bb7ba0dd3deec (127.23.218.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.684911 2784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.684990 2784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:07.684968 2786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.685047 2786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:07.685097 2926 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.685163 2926 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:52:07.693423 2958 master.cc:552] Master@127.23.218.252:43931 shutdown complete.
I20260809 18:52:07.693470 2958 master_runner.cc:305] Clearing existing system tablet
I20260809 18:52:07.693589 2958 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:07.693632 2958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:07.694239 2958 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:07.694722 3093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.694839 2958 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:07.694896 2958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.694921 2958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal
uuid: "1e4676ef914b45dd963ce1983aa14032"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "c3e2eab570c6a0325a7f44dc3873c82f"
server_key_iv: "e6b016d16c589f7dfa07b95a0db689d0"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.695654 2958 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:07.696856 2958 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 18:52:07.696928 2958 log.cc:1196] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 18:52:07.697121 2958 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: Deleting consensus metadata
I20260809 18:52:07.697229 2958 master_runner.cc:315] Copying system tablet from 127.23.218.254:38275
I20260809 18:52:07.697811 2958 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: tablet copy: Beginning tablet copy session from remote peer at address 127.23.218.254:38275
I20260809 18:52:07.705915 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.698092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55122 (local address 127.23.218.254:38275)
0809 18:52:07.698142 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.698144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.698244 (+ 100us) server_negotiation.cc:407] Connection header received
0809 18:52:07.698296 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.698296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.698306 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.698386 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.698667 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.699101 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.699748 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.699923 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.699971 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.703806 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.703807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.703807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.703813 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.704977 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.705289 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.705290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.705290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.705336 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.705495 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.705495 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.705496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.705586 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.705642 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.705771 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.705832 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 18:52:07.705940 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.698036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55122)
0809 18:52:07.698183 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.698192 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.698244 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.698396 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.698397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.698538 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.698602 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.698604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.699165 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.699166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.699682 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.699683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.699977 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.699978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.700038 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.703472 (+ 3434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.703476 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.703725 (+ 249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.705050 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.705050 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.705051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.705207 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.705389 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.705389 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.705390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.705445 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.705641 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.705642 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.705683 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.705741 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.705833 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260809 18:52:07.706260 1929 tablet_copy_service.cc:140] P 019912aa2162420e90731caaf172ec6b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1e4676ef914b45dd963ce1983aa14032 ({username='oryx', principal='oryx/127.23.218.252@KRBTEST.COM'} at 127.0.0.1:55122)
I20260809 18:52:07.706306 1929 tablet_copy_service.cc:161] P 019912aa2162420e90731caaf172ec6b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1e4676ef914b45dd963ce1983aa14032 at {username='oryx', principal='oryx/127.23.218.252@KRBTEST.COM'} at 127.0.0.1:55122: session id = 1e4676ef914b45dd963ce1983aa14032-00000000000000000000000000000000
I20260809 18:52:07.706691 1929 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 18:52:07.707108 2958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 18:52:07.708156 2958 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: tablet copy: Starting download of 0 data blocks...
I20260809 18:52:07.708293 2958 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: tablet copy: Starting download of 1 WAL segments...
I20260809 18:52:07.708977 2958 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 18:52:07.709779 1929 tablet_copy_service.cc:342] P 019912aa2162420e90731caaf172ec6b: Request end of tablet copy session 1e4676ef914b45dd963ce1983aa14032-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.218.252@KRBTEST.COM'} at 127.0.0.1:55122
I20260809 18:52:07.709834 1929 tablet_copy_service.cc:433] P 019912aa2162420e90731caaf172ec6b: ending tablet copy session 1e4676ef914b45dd963ce1983aa14032-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1e4676ef914b45dd963ce1983aa14032
I20260809 18:52:07.711853 2958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:07.712775 3102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.712780 3103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.715063 3108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:07.715174 2958 server_base.cc:1034] running on GCE node
I20260809 18:52:07.715274 2958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:07.715391 2958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:07.716495 2958 hybrid_clock.cc:648] HybridClock initialized: now 1786301527716487 us; error 21 us; skew 500 ppm
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:07.717725 2958 init.cc:377] Logged in from keytab as oryx/127.23.218.252@KRBTEST.COM (short username oryx)
I20260809 18:52:07.717968 2958 webserver.cc:460] Webserver started at http://127.23.218.252:45333/ using document root <none> and password file <none>
I20260809 18:52:07.718075 2958 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:07.718107 2958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:07.718784 2958 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:07.719267 3119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.719379 2958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:07.719416 2958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal
uuid: "1e4676ef914b45dd963ce1983aa14032"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "c3e2eab570c6a0325a7f44dc3873c82f"
server_key_iv: "e6b016d16c589f7dfa07b95a0db689d0"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.719465 2958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:07.732909 2958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:07.734205 2958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:07.736805 2958 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.252:43931
I20260809 18:52:07.736907 3207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.252:43931 every 8 connection(s)
I20260809 18:52:07.737082 2958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
I20260809 18:52:07.737615 3208 sys_catalog.cc:263] Verifying existing consensus state
I20260809 18:52:07.737962 3208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: Bootstrap starting.
I20260809 18:52:07.740146 3208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:52:07.740334 3208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: Bootstrap complete.
I20260809 18:52:07.740484 3208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } }
I20260809 18:52:07.740545 3208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1e4676ef914b45dd963ce1983aa14032, State: Initialized, Role: LEARNER
I20260809 18:52:07.740568 3208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } }
I20260809 18:52:07.740746 3208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:07.740772 3213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:07.740883 3213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: This master's current role is: LEARNER
I20260809 18:52:07.742026 3236 catalog_manager.cc:1259] Loaded cluster ID: 6cd7b2d124fd44a5817b7d4d2d228574
I20260809 18:52:07.742065 3236 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: loading cluster ID for follower catalog manager: success
I20260809 18:52:07.743116 3236 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: acquiring CA information for follower catalog manager: success
I20260809 18:52:07.743362 3236 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 18:52:07.786069 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.781026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.23.218.252:43931)
0809 18:52:07.781127 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.781129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.781136 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:07.781156 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.781157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.781166 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.781226 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.781683 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.782278 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.782923 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.783114 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.783165 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.784169 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.784170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.784170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.784175 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.785266 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.785549 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.785549 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.785550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.785592 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.785714 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.785715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.785715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.785799 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.785861 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.785932 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.785988 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":55,"threads_started":1}
I20260809 18:52:07.786577 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 2002
I20260809 18:52:07.801955 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:41941
--local_ip_for_outbound_sockets=127.23.218.193
--tserver_master_addrs=127.23.218.254:38275
--webserver_port=46793
--webserver_interface=127.23.218.193
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab
--principal=oryx/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.218.254:38275,127.23.218.253:39781
--tserver_master_addrs=127.23.218.254:38275,127.23.218.253:39781,127.23.218.252:43931 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:07.891163 3238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:07.891336 3238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:07.891356 3238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:07.892411 3238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:07.892445 3238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:07.892469 3238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:07.893843 3238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab
--principal=oryx/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:41941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=46793
--tserver_master_addrs=127.23.218.254:38275,127.23.218.253:39781,127.23.218.252:43931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:07.894006 3238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:07.894164 3238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:07.898628 3245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:07.898649 3244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:07.898733 3238 server_base.cc:1034] running on GCE node
W20260809 18:52:07.898691 3250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:07.899017 3238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:07.899226 3238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:07.900360 3238 hybrid_clock.cc:648] HybridClock initialized: now 1786301527900352 us; error 34 us; skew 500 ppm
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:07.902566 3238 init.cc:377] Logged in from keytab as oryx/127.23.218.193@KRBTEST.COM (short username oryx)
I20260809 18:52:07.902979 3238 webserver.cc:460] Webserver started at http://127.23.218.193:46793/ using document root <none> and password file <none>
I20260809 18:52:07.903172 3238 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:07.903228 3238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:07.904404 3238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:07.904886 3260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.905032 3238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.905112 3238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "d66a13491b2641e487b7554f63b90125"
format_stamp: "Formatted at 2026-08-09 18:52:06 on dist-test-slave-6zbq"
server_key: "9c89fe019c13c2adc9efc3fd7de83d42"
server_key_iv: "29dc686ef52acde441ef2f56cb8dd028"
server_key_version: "encryptionkey@0"
I20260809 18:52:07.905400 3238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:07.920404 3238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:07.922322 3238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:07.922461 3238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:07.922643 3238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:07.922909 3238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:07.922940 3238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.922960 3238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:07.922982 3238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:07.928180 3238 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:41941
I20260809 18:52:07.928241 3442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:41941 every 8 connection(s)
I20260809 18:52:07.928529 3238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.253@KRBTEST.COM
Aug 09 18:52:07 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301527, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.252@KRBTEST.COM
I20260809 18:52:07.937618 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 3238
I20260809 18:52:07.937772 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 2202
I20260809 18:52:07.938658 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.929231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:50355 (local address 127.23.218.254:38275)
0809 18:52:07.929283 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.929285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.929755 (+ 470us) server_negotiation.cc:407] Connection header received
0809 18:52:07.930195 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.930196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.930208 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.930261 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.931038 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.931516 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.932846 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.933033 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.933083 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.935480 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.935482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.935482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.935489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.936965 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.937411 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.937412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.937412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.937495 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.937870 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.937871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.937871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.938004 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.938042 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.938464 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.938557 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 18:52:07.938884 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.929421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:51231 (local address 127.23.218.252:43931)
0809 18:52:07.929462 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.929464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.929600 (+ 136us) server_negotiation.cc:407] Connection header received
0809 18:52:07.930114 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.930116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.930131 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.930178 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.931046 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.931782 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.933586 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.933907 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.933967 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.936450 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.936451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.936451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.936459 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.937891 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.938219 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.938220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.938220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.938268 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.938440 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.938440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.938441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.938542 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.938573 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.938731 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.938802 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:52:07.939265 3458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.929128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.23.218.193:51231)
0809 18:52:07.929513 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.929528 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.929957 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.930237 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.930241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.930748 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.930928 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.930934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.932864 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.932865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.933510 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.933513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.934016 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.934017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.934068 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.934412 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.934416 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.936311 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.937983 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.937984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.937984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.938148 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.938322 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.938323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.938323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.938387 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.938605 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.938605 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.938637 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.938959 (+ 322us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.939098 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":33,"thread_start_us":71,"threads_started":1}
I20260809 18:52:07.939818 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.929128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.193:50355)
0809 18:52:07.929688 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.929696 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.929971 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.930283 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.930284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.930702 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.930928 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.930934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.931614 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.931615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.932765 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.932768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.933107 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.933108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.933238 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.933830 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.933837 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.935398 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.937034 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.937042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.937044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.937336 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.937567 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.937568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.937568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.937754 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.938085 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.938104 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.938254 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.939128 (+ 874us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.939244 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":204,"thread_start_us":57,"threads_started":1}
I20260809 18:52:07.941356 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.929127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.23.218.193:43093)
0809 18:52:07.929489 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:07.929526 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:07.929963 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:07.930519 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:07.930520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:07.930746 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:07.930927 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.930931 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.931863 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.931864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.932765 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:07.932768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.933383 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.933384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:07.933447 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.933960 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:07.933964 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:07.935821 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:07.937872 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.937873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.937874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.940462 (+ 2588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.940708 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:07.940709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:07.940709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:07.940763 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:07.941125 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:07.941126 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:07.941158 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 18:52:07.941201 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 18:52:07.941271 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":18,"thread_start_us":90,"threads_started":1}
I20260809 18:52:07.941444 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:07.929171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:43093 (local address 127.23.218.253:39781)
0809 18:52:07.929218 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:07.929220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:07.929627 (+ 407us) server_negotiation.cc:407] Connection header received
0809 18:52:07.930049 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:07.930050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:07.930064 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:07.930133 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:07.931010 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.931790 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.933130 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:07.933310 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:07.933361 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:07.936163 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:07.936164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:07.936164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:07.936173 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:07.937778 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.940564 (+ 2786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.940565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.940566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.940641 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:07.940831 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:07.940832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:07.940832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:07.941041 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:07.941141 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:07.941255 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 18:52:07.941345 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:52:07.941846 3450 heartbeater.cc:347] Connected to a master server at 127.23.218.253:39781
I20260809 18:52:07.941936 3450 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:07.942109 3450 heartbeater.cc:511] Master 127.23.218.253:39781 requested a full tablet report, sending...
I20260809 18:52:07.943657 2854 ts_manager.cc:194] Registered new tserver with Master: d66a13491b2641e487b7554f63b90125 (127.23.218.193:41941)
I20260809 18:52:07.958761 3445 heartbeater.cc:347] Connected to a master server at 127.23.218.252:43931
I20260809 18:52:07.958818 3445 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:07.958881 3445 heartbeater.cc:511] Master 127.23.218.252:43931 requested a full tablet report, sending...
I20260809 18:52:07.959118 3448 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38275
I20260809 18:52:07.959163 3448 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:07.959218 3448 heartbeater.cc:511] Master 127.23.218.254:38275 requested a full tablet report, sending...
I20260809 18:52:07.959213 3142 ts_manager.cc:194] Registered new tserver with Master: d66a13491b2641e487b7554f63b90125 (127.23.218.193:41941)
I20260809 18:52:07.959472 1897 ts_manager.cc:194] Re-registered known tserver with Master: d66a13491b2641e487b7554f63b90125 (127.23.218.193:41941)
I20260809 18:52:07.959929 1897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:50355
I20260809 18:52:07.962795 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:44607
--local_ip_for_outbound_sockets=127.23.218.194
--tserver_master_addrs=127.23.218.254:38275
--webserver_port=36505
--webserver_interface=127.23.218.194
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab
--principal=oryx/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.218.254:38275,127.23.218.253:39781
--tserver_master_addrs=127.23.218.254:38275,127.23.218.253:39781,127.23.218.252:43931 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:08.041610 3460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:08.041776 3460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:08.041801 3460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:08.042977 3460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:08.043015 3460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:08.043038 3460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:08.044502 3460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab
--principal=oryx/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:44607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=36505
--tserver_master_addrs=127.23.218.254:38275,127.23.218.253:39781,127.23.218.252:43931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:08.044684 3460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:08.044875 3460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:08.049482 3467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:08.049489 3472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:08.049559 3460 server_base.cc:1034] running on GCE node
W20260809 18:52:08.049489 3466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:08.049836 3460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:08.050027 3460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:08.051159 3460 hybrid_clock.cc:648] HybridClock initialized: now 1786301528051144 us; error 29 us; skew 500 ppm
Aug 09 18:52:08 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301528, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:08.053400 3460 init.cc:377] Logged in from keytab as oryx/127.23.218.194@KRBTEST.COM (short username oryx)
I20260809 18:52:08.053743 3460 webserver.cc:460] Webserver started at http://127.23.218.194:36505/ using document root <none> and password file <none>
I20260809 18:52:08.053931 3460 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:08.053980 3460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:08.055172 3460 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:08.055662 3482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:08.055801 3460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:08.055881 3460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "ea463fa92324405a9cad02790dd58f8f"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "7a2b9d26daba36c5098d6b8aa9455997"
server_key_iv: "1773078f63404c384e7286046a434a8b"
server_key_version: "encryptionkey@0"
I20260809 18:52:08.056143 3460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:08.073413 3460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:08.075372 3460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:08.075500 3460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:08.075671 3460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:08.075929 3460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:08.075961 3460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:08.075987 3460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:08.076004 3460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:08.081292 3460 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:44607
I20260809 18:52:08.081346 3678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:44607 every 8 connection(s)
I20260809 18:52:08.081631 3460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:52:08.087433 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 3460
I20260809 18:52:08.087601 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 2423
Aug 09 18:52:08 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301528, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.252@KRBTEST.COM
Aug 09 18:52:08 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301528, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
Aug 09 18:52:08 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301528, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.253@KRBTEST.COM
I20260809 18:52:08.093972 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.082429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:58409 (local address 127.23.218.252:43931)
0809 18:52:08.082498 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.082500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.082648 (+ 148us) server_negotiation.cc:407] Connection header received
0809 18:52:08.083595 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.083596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.083610 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.083661 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.084299 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.084828 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.086118 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.086367 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.086435 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.089034 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.089036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.089036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.089043 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.092482 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.092996 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.092996 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.092997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.093045 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.093333 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.093333 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.093334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.093430 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.093475 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.093795 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.093879 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 18:52:08.094270 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.082254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.23.218.194:58409)
0809 18:52:08.082550 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.082572 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.083496 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.083728 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.083731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.084030 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:08.084211 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.084215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.084926 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.084927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.086010 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.086012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.086440 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.086441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.086504 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.087056 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:08.087060 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:08.088943 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:08.092564 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.092573 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.092576 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.092924 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.093106 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.093107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.093107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.093268 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.093486 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:08.093494 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:08.093659 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.093771 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.093997 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":113,"threads_started":1}
I20260809 18:52:08.094954 3680 heartbeater.cc:347] Connected to a master server at 127.23.218.252:43931
I20260809 18:52:08.095064 3680 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:08.095286 3680 heartbeater.cc:511] Master 127.23.218.252:43931 requested a full tablet report, sending...
I20260809 18:52:08.096040 3142 ts_manager.cc:194] Registered new tserver with Master: ea463fa92324405a9cad02790dd58f8f (127.23.218.194:44607)
I20260809 18:52:08.096508 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.082799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:43747 (local address 127.23.218.253:39781)
0809 18:52:08.082849 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.082851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.082859 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:08.083615 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.083616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.083629 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.083660 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.084312 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.085097 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.085935 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.086195 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.086270 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.091337 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.091338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.091338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.091345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.095179 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.095665 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.095666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.095667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.095724 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.095935 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.095936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.095936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.096059 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.096111 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.096349 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.096423 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 18:52:08.096819 3686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.082253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.194:40535)
0809 18:52:08.082532 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.082573 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.083509 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.083737 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.083738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.084034 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:08.084212 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.084218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.085174 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.085174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.085975 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.085978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.086492 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.086493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.086544 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.087137 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:08.087141 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:08.094025 (+ 6884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:08.095798 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.095798 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.095799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.096000 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.096190 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.096191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.096191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.096244 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.096508 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:08.096509 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:08.096547 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.096617 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.096715 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":7,"thread_start_us":104,"threads_started":1}
I20260809 18:52:08.096903 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.082692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.23.218.194:43747)
0809 18:52:08.082803 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.082812 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.083509 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.083741 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.083742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.084033 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:08.084212 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.084218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.085216 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.085217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.085863 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.085865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.086287 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.086288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.086449 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.086869 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:08.086880 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:08.091253 (+ 4373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:08.095381 (+ 4128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.095382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.095383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.095575 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.095795 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.095796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.095796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.095862 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.096199 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:08.096200 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:08.096239 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.096627 (+ 388us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.096780 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":100,"thread_start_us":77,"threads_started":1}
I20260809 18:52:08.097165 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.082245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:40535 (local address 127.23.218.254:38275)
0809 18:52:08.082299 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.082301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.082647 (+ 346us) server_negotiation.cc:407] Connection header received
0809 18:52:08.083642 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.083643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.083653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.083696 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.084310 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.085104 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.086061 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.086385 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.086442 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.094120 (+ 7678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.094121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.094122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.094129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.095727 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.096070 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.096071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.096072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.096126 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.096307 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.096308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.096309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.096427 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.096483 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.097004 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.097081 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:52:08.097424 3681 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38275
I20260809 18:52:08.097429 3682 heartbeater.cc:347] Connected to a master server at 127.23.218.253:39781
I20260809 18:52:08.097474 3681 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:08.097487 3682 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:08.097534 3681 heartbeater.cc:511] Master 127.23.218.254:38275 requested a full tablet report, sending...
I20260809 18:52:08.097534 3682 heartbeater.cc:511] Master 127.23.218.253:39781 requested a full tablet report, sending...
I20260809 18:52:08.097760 1897 ts_manager.cc:194] Re-registered known tserver with Master: ea463fa92324405a9cad02790dd58f8f (127.23.218.194:44607)
I20260809 18:52:08.097889 2854 ts_manager.cc:194] Registered new tserver with Master: ea463fa92324405a9cad02790dd58f8f (127.23.218.194:44607)
I20260809 18:52:08.098129 1897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:40535
I20260809 18:52:08.108369 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:35715
--local_ip_for_outbound_sockets=127.23.218.195
--tserver_master_addrs=127.23.218.254:38275
--webserver_port=43329
--webserver_interface=127.23.218.195
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab
--principal=oryx/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.218.254:38275,127.23.218.253:39781
--tserver_master_addrs=127.23.218.254:38275,127.23.218.253:39781,127.23.218.252:43931 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:52:08.169304 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.166270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.0.0.1:35738)
0809 18:52:08.166320 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.166332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.166405 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.166551 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.166552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.166553 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:08.166597 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.166598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.167496 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.167497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.168248 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.168251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.169090 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.169091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.169152 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.169156 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.169197 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.169237 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 18:52:08.169324 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.166365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.23.218.252:43931)
0809 18:52:08.166413 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.166415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.166425 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:52:08.166452 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.166453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.166476 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.166544 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:08.166649 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.167417 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.168318 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.168979 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.169086 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.169152 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.169203 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.169256 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:52:08.169718 3155 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 18:52:08.169878 3213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 019912aa2162420e90731caaf172ec6b. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:08.169986 3213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: This master's current role is: LEARNER
W20260809 18:52:08.189810 3693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:08.189965 3693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:08.189986 3693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:08.191098 3693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:08.191136 3693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:08.191160 3693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:08.192565 3693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab
--principal=oryx/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:35715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=43329
--tserver_master_addrs=127.23.218.254:38275,127.23.218.253:39781,127.23.218.252:43931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:08.192739 3693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:08.192898 3693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:08.197264 3699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:08.197374 3693 server_base.cc:1034] running on GCE node
W20260809 18:52:08.197248 3700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:08.197239 3702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:08.197656 3693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:08.197856 3693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:08.198984 3693 hybrid_clock.cc:648] HybridClock initialized: now 1786301528198971 us; error 28 us; skew 500 ppm
Aug 09 18:52:08 dist-test-slave-6zbq krb5kdc[1830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301528, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:08.201236 3693 init.cc:377] Logged in from keytab as oryx/127.23.218.195@KRBTEST.COM (short username oryx)
I20260809 18:52:08.201627 3693 webserver.cc:460] Webserver started at http://127.23.218.195:43329/ using document root <none> and password file <none>
I20260809 18:52:08.201807 3693 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:08.201853 3693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:08.203048 3693 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 18:52:08.203532 3712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:08.203677 3693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:08.203750 3693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "01c2bc35b7ee44a8ab3386e497dae5e3"
format_stamp: "Formatted at 2026-08-09 18:52:07 on dist-test-slave-6zbq"
server_key: "fbe1b28bec055a9d67216c1bf12d479d"
server_key_iv: "73a513d1ce7afb693b41b754eb06a854"
server_key_version: "encryptionkey@0"
I20260809 18:52:08.204016 3693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:08.218549 3693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:08.220182 3693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:08.220289 3693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:08.220489 3693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:08.220763 3693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:08.220800 3693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:08.220829 3693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:08.220852 3693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:08.227201 3693 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:35715
I20260809 18:52:08.227257 3903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:35715 every 8 connection(s)
I20260809 18:52:08.227519 3693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:52:08.233085 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 3693
Aug 09 18:52:08 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301528, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
Aug 09 18:52:08 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301528, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.253@KRBTEST.COM
Aug 09 18:52:08 dist-test-slave-6zbq krb5kdc[1830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301528, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.252@KRBTEST.COM
I20260809 18:52:08.238176 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.228181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.23.218.195:45627)
0809 18:52:08.228537 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.228545 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.228947 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.229180 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.229180 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.229641 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:08.229817 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.229823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.230566 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.230566 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.231732 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.231735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.232235 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.232236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.232323 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.232709 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:08.232717 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:08.235013 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:08.236676 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.236684 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.236686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.236957 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.237165 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.237166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.237166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.237293 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.237557 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:08.237564 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:08.237723 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.237827 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.237977 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":7,"thread_start_us":78,"threads_started":1}
I20260809 18:52:08.238483 2951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.228439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:45627 (local address 127.23.218.253:39781)
0809 18:52:08.228481 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.228483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.228586 (+ 103us) server_negotiation.cc:407] Connection header received
0809 18:52:08.229053 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.229054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.229066 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.229104 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.229922 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.230447 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.231818 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.232130 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.232183 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.235127 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.235129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.235129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.235137 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.236584 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.237038 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.237039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.237039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.237098 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.237357 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.237358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.237358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.237474 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.237531 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.238301 (+ 770us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.238395 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:52:08.238554 1981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.228427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:53135 (local address 127.23.218.254:38275)
0809 18:52:08.228471 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.228472 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.228585 (+ 113us) server_negotiation.cc:407] Connection header received
0809 18:52:08.229053 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.229055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.229066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.229096 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.229930 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.230678 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.231817 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.232116 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.232180 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.235671 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.235673 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.235673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.235680 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.237399 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.237758 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.237759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.237760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.237811 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.238006 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.238007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.238008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.238127 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.238167 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.238393 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.238466 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:52:08.238775 3910 heartbeater.cc:347] Connected to a master server at 127.23.218.253:39781
I20260809 18:52:08.238863 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.228179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.195:53135)
0809 18:52:08.228460 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.228498 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.228941 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.229171 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.229176 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.229629 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:08.229817 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.229823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.230746 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.230747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.231732 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.231735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.232192 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.232193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.232325 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.232861 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:08.232866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:08.235607 (+ 2741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:08.237474 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.237475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.237475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.237682 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.237880 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.237881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.237881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.237936 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.238217 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:08.238218 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:08.238258 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.238634 (+ 376us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.238729 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":5,"thread_start_us":76,"threads_started":1}
I20260809 18:52:08.239336 3906 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38275
I20260809 18:52:08.239382 3906 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:08.239421 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.228339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:57687 (local address 127.23.218.252:43931)
0809 18:52:08.228388 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.228390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.228622 (+ 232us) server_negotiation.cc:407] Connection header received
0809 18:52:08.229017 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.229019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.229031 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.229076 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.229895 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.230671 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.232539 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.232821 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.232901 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.235377 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.235378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.235378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.235384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.237135 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.237487 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.237488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.237488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.237537 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.238779 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.238780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.238780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.238904 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.238949 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.239259 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.239333 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:52:08.239418 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.228181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.23.218.195:57687)
0809 18:52:08.228552 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.228560 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.228950 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.229386 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.229387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.229651 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:08.229816 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.229820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.231825 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.231825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.232422 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.232424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.232894 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.232895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.232958 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.233318 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:08.233323 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:08.235306 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:08.237221 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.237222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.237222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.237412 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.237607 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:08.237607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:08.237608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:08.237661 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:08.238995 (+ 1334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:08.238995 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:08.239027 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.239082 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.239177 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":215,"thread_start_us":59,"threads_started":1}
I20260809 18:52:08.239543 3906 heartbeater.cc:511] Master 127.23.218.254:38275 requested a full tablet report, sending...
I20260809 18:52:08.239732 3904 heartbeater.cc:347] Connected to a master server at 127.23.218.252:43931
I20260809 18:52:08.239773 3904 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:08.239820 3904 heartbeater.cc:511] Master 127.23.218.252:43931 requested a full tablet report, sending...
I20260809 18:52:08.238871 3910 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:08.239933 3910 heartbeater.cc:511] Master 127.23.218.253:39781 requested a full tablet report, sending...
I20260809 18:52:08.240144 1897 ts_manager.cc:194] Re-registered known tserver with Master: 01c2bc35b7ee44a8ab3386e497dae5e3 (127.23.218.195:35715)
I20260809 18:52:08.240193 3142 ts_manager.cc:194] Registered new tserver with Master: 01c2bc35b7ee44a8ab3386e497dae5e3 (127.23.218.195:35715)
I20260809 18:52:08.240193 2854 ts_manager.cc:194] Registered new tserver with Master: 01c2bc35b7ee44a8ab3386e497dae5e3 (127.23.218.195:35715)
I20260809 18:52:08.240568 1897 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:53135
I20260809 18:52:08.245975 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.234675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.23.218.252:43931)
0809 18:52:08.236078 (+ 1403us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.236080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.236086 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:52:08.236114 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.236115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.236126 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.236187 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.238179 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.239783 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.241670 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.241977 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.242029 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.243498 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.243499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.243500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.243506 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.244946 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.245242 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.245243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.245243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.245293 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.245493 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.245494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.245494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.245618 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.245708 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.245815 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.245885 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1385,"thread_start_us":53,"threads_started":1}
I20260809 18:52:08.246764 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.234385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.23.218.253:39781)
0809 18:52:08.235829 (+ 1444us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.235831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.235836 (+ 5us) server_negotiation.cc:407] Connection header received
0809 18:52:08.235852 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.235852 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.235861 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.237266 (+ 1405us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.237298 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.239351 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.242723 (+ 3372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.242906 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.242956 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.244143 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.244145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.244145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.244152 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.245556 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.245987 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.245988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.245989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.246046 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.246259 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.246260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.246260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.246402 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.246478 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.246620 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.246681 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1429,"thread_start_us":55,"threads_started":1}
I20260809 18:52:08.247646 3924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.234143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.23.218.254:38275)
0809 18:52:08.236232 (+ 2089us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.236233 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.236240 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:08.236258 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.236259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.236268 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.236335 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.240486 (+ 4151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.241114 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.242108 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.242315 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.242404 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.244562 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.244564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.244564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.244572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.246643 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.246980 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.246981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.246982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.247042 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.247206 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.247207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.247207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.247310 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.247389 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.247486 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.247560 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1392,"thread_start_us":66,"threads_started":1}
I20260809 18:52:08.248817 1897 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:52:08.250262 1897 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:08.250910 3089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: NON_VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: true } } }
I20260809 18:52:08.251000 3089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:08.252245 3089 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: attempting to promote NON_VOTER 1e4676ef914b45dd963ce1983aa14032 to VOTER
I20260809 18:52:08.252370 3089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } }
I20260809 18:52:08.252820 2873 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Refusing update from remote peer 019912aa2162420e90731caaf172ec6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260809 18:52:08.253139 3154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 1 LEARNER]: Refusing update from remote peer 019912aa2162420e90731caaf172ec6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260809 18:52:08.253369 3945 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 18:52:08.253459 3936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 18:52:08.254117 3936 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 1e4676ef914b45dd963ce1983aa14032 (127.23.218.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:08.254333 2873 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 1e4676ef914b45dd963ce1983aa14032 (127.23.218.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:08.254304 3154 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 1e4676ef914b45dd963ce1983aa14032 (127.23.218.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:08.255056 3936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:08.255131 3936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:08.255179 3936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:08.255219 3936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:08.255254 3936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:08.255296 3936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:08.255265 3213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:08.255327 3213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:52:08.255455 3950 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "019912aa2162420e90731caaf172ec6b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:08.255534 3950 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:52:08.259692 3953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.255891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:35715 (local address 127.0.0.1:39640)
0809 18:52:08.256107 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.256114 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.256148 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.256425 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.256425 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.256427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:08.256454 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.256455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.257554 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.257555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.258685 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.258686 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.259461 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.259462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.259535 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.259539 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.259568 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.259623 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260809 18:52:08.259778 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.255851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:44607 (local address 127.0.0.1:52726)
0809 18:52:08.255990 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.256001 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.256037 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.256636 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.256637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.256638 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:08.256667 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.256668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.257869 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.257870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.258728 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.258731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.259571 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.259572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.259631 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.259634 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.259672 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.259702 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260809 18:52:08.259807 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.256040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.23.218.194:44607)
0809 18:52:08.256495 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.256499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.256507 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:08.256541 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.256543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.256558 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.256632 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:08.256730 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.257802 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.258799 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.259471 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.259592 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.259664 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.259689 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.259728 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":85,"threads_started":1}
I20260809 18:52:08.259859 3952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.255811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41941 (local address 127.0.0.1:48144)
0809 18:52:08.255961 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.255972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.256026 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.256383 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.256383 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.256386 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:08.256425 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.256427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.257480 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.257481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.258651 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.258654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.259654 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.259655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.259715 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.259719 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.259746 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.259801 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":1}
I20260809 18:52:08.260233 3956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.255908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.23.218.193:41941)
0809 18:52:08.256224 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.256228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.256237 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:08.256277 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.256279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.256296 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.256377 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:08.256518 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.257412 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.258759 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.259570 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.259960 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.260005 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.260026 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.260056 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":168,"threads_started":1}
I20260809 18:52:08.260665 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.256166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39640 (local address 127.23.218.195:35715)
0809 18:52:08.256296 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.256300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.256307 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:08.256337 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.256339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.256351 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.256416 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:08.256507 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.257485 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.258741 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.259269 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.260473 (+ 1204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.260538 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.260563 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.260598 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260809 18:52:08.260772 3570 tablet_service.cc:1467] Processing CreateTablet for tablet 585a586667db400ba30bed6fedfa503c (DEFAULT_TABLE table=test-table [id=7b235538013045458fedd8df380dec03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:08.260897 3345 tablet_service.cc:1467] Processing CreateTablet for tablet 585a586667db400ba30bed6fedfa503c (DEFAULT_TABLE table=test-table [id=7b235538013045458fedd8df380dec03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:08.261140 3570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585a586667db400ba30bed6fedfa503c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:08.261263 3345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585a586667db400ba30bed6fedfa503c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:08.261214 3798 tablet_service.cc:1467] Processing CreateTablet for tablet 585a586667db400ba30bed6fedfa503c (DEFAULT_TABLE table=test-table [id=7b235538013045458fedd8df380dec03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:08.261497 3798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585a586667db400ba30bed6fedfa503c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:08.263823 3960 tablet_bootstrap.cc:492] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: Bootstrap starting.
I20260809 18:52:08.264370 3960 tablet_bootstrap.cc:654] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:08.264616 3961 tablet_bootstrap.cc:492] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: Bootstrap starting.
I20260809 18:52:08.264695 3960 log.cc:824] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:08.264937 3962 tablet_bootstrap.cc:492] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: Bootstrap starting.
I20260809 18:52:08.265424 3961 tablet_bootstrap.cc:654] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:08.265429 3962 tablet_bootstrap.cc:654] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:08.265666 3961 log.cc:824] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:08.265686 3962 log.cc:824] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:08.266425 3961 tablet_bootstrap.cc:492] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: No bootstrap required, opened a new log
I20260809 18:52:08.266489 3961 ts_tablet_manager.cc:1397] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:52:08.266527 3962 tablet_bootstrap.cc:492] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: No bootstrap required, opened a new log
I20260809 18:52:08.266527 3960 tablet_bootstrap.cc:492] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: No bootstrap required, opened a new log
I20260809 18:52:08.266595 3960 ts_tablet_manager.cc:1397] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 18:52:08.266602 3962 ts_tablet_manager.cc:1397] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:52:08.267899 3961 raft_consensus.cc:348] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } }
I20260809 18:52:08.268016 3961 raft_consensus.cc:374] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:08.268049 3961 raft_consensus.cc:729] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d66a13491b2641e487b7554f63b90125, State: Initialized, Role: FOLLOWER
I20260809 18:52:08.268246 3961 consensus_queue.cc:260] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } }
I20260809 18:52:08.268500 3961 ts_tablet_manager.cc:1428] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 18:52:08.268813 3448 heartbeater.cc:503] Master 127.23.218.254:38275 was elected leader, sending a full tablet report...
I20260809 18:52:08.268958 3962 raft_consensus.cc:348] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } }
I20260809 18:52:08.269112 3962 raft_consensus.cc:374] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:08.269155 3962 raft_consensus.cc:729] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01c2bc35b7ee44a8ab3386e497dae5e3, State: Initialized, Role: FOLLOWER
I20260809 18:52:08.269250 3962 consensus_queue.cc:260] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } }
I20260809 18:52:08.269497 3962 ts_tablet_manager.cc:1428] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 18:52:08.269563 3906 heartbeater.cc:503] Master 127.23.218.254:38275 was elected leader, sending a full tablet report...
I20260809 18:52:08.270447 3960 raft_consensus.cc:348] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } }
I20260809 18:52:08.270560 3960 raft_consensus.cc:374] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:08.270577 3960 raft_consensus.cc:729] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea463fa92324405a9cad02790dd58f8f, State: Initialized, Role: FOLLOWER
I20260809 18:52:08.270663 3960 consensus_queue.cc:260] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } }
I20260809 18:52:08.270884 3960 ts_tablet_manager.cc:1428] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260809 18:52:08.270927 3681 heartbeater.cc:503] Master 127.23.218.254:38275 was elected leader, sending a full tablet report...
W20260809 18:52:08.332664 3683 tablet.cc:2369] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:52:08.353392 3971 raft_consensus.cc:482] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:08.353482 3971 raft_consensus.cc:504] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } }
I20260809 18:52:08.353951 3971 leader_election.cc:288] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea463fa92324405a9cad02790dd58f8f (127.23.218.194:44607), 01c2bc35b7ee44a8ab3386e497dae5e3 (127.23.218.195:35715)
I20260809 18:52:08.356634 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.354200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:58991 (local address 127.23.218.195:35715)
0809 18:52:08.354267 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.354269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.354277 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:08.354302 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.354303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.354313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.354398 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:08.354544 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.355021 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.355932 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.356320 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.356374 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.356381 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.356529 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.356569 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 18:52:08.356650 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.354277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:50159 (local address 127.23.218.194:44607)
0809 18:52:08.354318 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.354319 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.354325 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:52:08.354362 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.354363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.354372 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.354420 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:08.354555 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.355005 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.355958 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.356349 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.356415 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.356423 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.356551 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.356589 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:52:08.356670 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.354079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:44607 (local address 127.23.218.193:50159)
0809 18:52:08.354123 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.354134 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.354175 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.354428 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.354429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.354431 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:08.354493 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.354494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.355074 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.355075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.355870 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.355872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.356427 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.356428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.356485 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.356488 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.356537 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.356573 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":24}
I20260809 18:52:08.356691 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.354130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:35715 (local address 127.23.218.193:58991)
0809 18:52:08.354194 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:08.354205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:08.354247 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:08.354406 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:08.354407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:08.354409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:08.354486 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.354487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.355093 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.355094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.355854 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:08.355856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.356393 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.356394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:08.356467 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.356470 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:08.356511 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 18:52:08.356550 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":3}
I20260809 18:52:08.356953 3604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585a586667db400ba30bed6fedfa503c" candidate_uuid: "d66a13491b2641e487b7554f63b90125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea463fa92324405a9cad02790dd58f8f" is_pre_election: true
I20260809 18:52:08.357012 3826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585a586667db400ba30bed6fedfa503c" candidate_uuid: "d66a13491b2641e487b7554f63b90125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" is_pre_election: true
I20260809 18:52:08.357086 3604 raft_consensus.cc:2454] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d66a13491b2641e487b7554f63b90125 in term 0.
I20260809 18:52:08.357153 3826 raft_consensus.cc:2454] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d66a13491b2641e487b7554f63b90125 in term 0.
I20260809 18:52:08.357295 3264 leader_election.cc:302] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d66a13491b2641e487b7554f63b90125, ea463fa92324405a9cad02790dd58f8f; no voters:
I20260809 18:52:08.357467 3971 raft_consensus.cc:2781] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:08.357518 3971 raft_consensus.cc:482] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:08.357534 3971 raft_consensus.cc:3037] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:08.358217 3971 raft_consensus.cc:504] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } }
I20260809 18:52:08.358371 3971 leader_election.cc:288] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [CANDIDATE]: Term 1 election: Requested vote from peers ea463fa92324405a9cad02790dd58f8f (127.23.218.194:44607), 01c2bc35b7ee44a8ab3386e497dae5e3 (127.23.218.195:35715)
I20260809 18:52:08.358508 3826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585a586667db400ba30bed6fedfa503c" candidate_uuid: "d66a13491b2641e487b7554f63b90125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3"
I20260809 18:52:08.358556 3604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585a586667db400ba30bed6fedfa503c" candidate_uuid: "d66a13491b2641e487b7554f63b90125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea463fa92324405a9cad02790dd58f8f"
I20260809 18:52:08.358587 3826 raft_consensus.cc:3037] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:08.358629 3604 raft_consensus.cc:3037] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:08.359438 3604 raft_consensus.cc:2454] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d66a13491b2641e487b7554f63b90125 in term 1.
I20260809 18:52:08.359438 3826 raft_consensus.cc:2454] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d66a13491b2641e487b7554f63b90125 in term 1.
I20260809 18:52:08.359618 3261 leader_election.cc:302] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01c2bc35b7ee44a8ab3386e497dae5e3, d66a13491b2641e487b7554f63b90125; no voters:
I20260809 18:52:08.359732 3971 raft_consensus.cc:2781] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:08.359864 3971 raft_consensus.cc:686] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 1 LEADER]: Becoming Leader. State: Replica: d66a13491b2641e487b7554f63b90125, State: Running, Role: LEADER
I20260809 18:52:08.359972 3971 consensus_queue.cc:237] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } }
I20260809 18:52:08.360805 1897 catalog_manager.cc:5495] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 reported cstate change: term changed from 0 to 1, leader changed from <none> to d66a13491b2641e487b7554f63b90125 (127.23.218.193). New cstate: current_term: 1 leader_uuid: "d66a13491b2641e487b7554f63b90125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } health_report { overall_health: UNKNOWN } } }
I20260809 18:52:08.373848 3956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.371670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48152 (local address 127.23.218.193:41941)
0809 18:52:08.371709 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.371711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.371719 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:08.371775 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.371776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.371793 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.371849 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:08.371972 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.372440 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.373172 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.373369 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.373421 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.373569 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:08.373669 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:08.373736 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.373773 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.373791 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:52:08.375366 3604 raft_consensus.cc:1261] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [term 1 FOLLOWER]: Refusing update from remote peer d66a13491b2641e487b7554f63b90125: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:08.375396 3826 raft_consensus.cc:1261] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 1 FOLLOWER]: Refusing update from remote peer d66a13491b2641e487b7554f63b90125: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:08.375653 3971 consensus_queue.cc:1035] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:08.375772 3974 consensus_queue.cc:1035] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:08.378762 3981 mvcc.cc:205] Tried to move back new op lower bound from 7316691060223537152 to 7316691060163002368. Current Snapshot: MvccSnapshot[applied={T|T < 7316691060223537152}]
I20260809 18:52:08.380789 3985 mvcc.cc:205] Tried to move back new op lower bound from 7316691060223537152 to 7316691060163002368. Current Snapshot: MvccSnapshot[applied={T|T < 7316691060223537152}]
I20260809 18:52:08.382107 1897 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:08.382189 1897 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:08.383654 1897 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 019912aa2162420e90731caaf172ec6b: Sending DeleteTablet for 3 replicas of tablet 585a586667db400ba30bed6fedfa503c
I20260809 18:52:08.383944 3570 tablet_service.cc:1514] Processing DeleteTablet for tablet 585a586667db400ba30bed6fedfa503c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:08 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:52726
I20260809 18:52:08.384007 3798 tablet_service.cc:1514] Processing DeleteTablet for tablet 585a586667db400ba30bed6fedfa503c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:08 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:39640
I20260809 18:52:08.384032 3345 tablet_service.cc:1514] Processing DeleteTablet for tablet 585a586667db400ba30bed6fedfa503c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:08 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:48144
I20260809 18:52:08.384243 3994 tablet_replica.cc:330] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: stopping tablet replica
I20260809 18:52:08.384339 3996 tablet_replica.cc:330] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: stopping tablet replica
I20260809 18:52:08.384474 3996 raft_consensus.cc:2229] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:52:08.384610 3996 raft_consensus.cc:2258] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:08.384799 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 1850
I20260809 18:52:08.385062 3996 ts_tablet_manager.cc:1905] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:08.386452 3996 ts_tablet_manager.cc:1918] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:08.386504 3996 log.cc:1196] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/585a586667db400ba30bed6fedfa503c
I20260809 18:52:08.386695 3996 ts_tablet_manager.cc:1939] T 585a586667db400ba30bed6fedfa503c P d66a13491b2641e487b7554f63b90125: Deleting consensus metadata
I20260809 18:52:08.395102 3994 raft_consensus.cc:2229] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:08.395246 3994 raft_consensus.cc:2258] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:08.395435 3998 tablet_replica.cc:330] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: stopping tablet replica
I20260809 18:52:08.395532 3998 raft_consensus.cc:2229] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:08.395627 3998 raft_consensus.cc:2258] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:08.396076 3998 ts_tablet_manager.cc:1905] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:08.396144 3994 ts_tablet_manager.cc:1905] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:08.397970 3994 ts_tablet_manager.cc:1918] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:08.398046 3994 log.cc:1196] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/585a586667db400ba30bed6fedfa503c
I20260809 18:52:08.398269 3998 ts_tablet_manager.cc:1918] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:08.398315 3994 ts_tablet_manager.cc:1939] T 585a586667db400ba30bed6fedfa503c P 01c2bc35b7ee44a8ab3386e497dae5e3: Deleting consensus metadata
I20260809 18:52:08.398447 3998 log.cc:1196] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/585a586667db400ba30bed6fedfa503c
I20260809 18:52:08.398741 3998 ts_tablet_manager.cc:1939] T 585a586667db400ba30bed6fedfa503c P ea463fa92324405a9cad02790dd58f8f: Deleting consensus metadata
W20260809 18:52:08.401015 3713 connection.cc:535] server connection from 127.0.0.1:39640 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 18:52:08.401033 3484 connection.cc:535] server connection from 127.0.0.1:52726 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 18:52:08.401153 3266 connection.cc:535] server connection from 127.0.0.1:48144 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 18:52:08.410392 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.403387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35758 (local address 127.23.218.252:43931)
0809 18:52:08.403425 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.403426 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.403853 (+ 427us) server_negotiation.cc:407] Connection header received
0809 18:52:08.403950 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.403951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.403965 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.403994 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.404792 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.405350 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.406414 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.406636 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.406694 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.407846 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.407848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.407848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.407855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.409290 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.409635 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.409636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.409636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.409694 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.409853 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.409854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.409854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.409960 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.410045 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.410190 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.410275 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:52:08.410637 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:08.402783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.23.218.253:39781)
0809 18:52:08.402823 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:08.402825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:08.403055 (+ 230us) server_negotiation.cc:407] Connection header received
0809 18:52:08.403148 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:08.403148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:08.403171 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:08.403201 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:08.403922 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.404699 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.405738 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:08.406061 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:08.406132 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:08.407635 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:08.407637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:08.407637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:08.407645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:08.409231 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.409682 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.409683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.409683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.409742 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:08.409931 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:08.409932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:08.409932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:08.410054 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:08.410122 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:08.410334 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 18:52:08.410576 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:52:09.379776 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.379513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.193:46007)
0809 18:52:09.379648 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.379724 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":52,"threads_started":1}
I20260809 18:52:09.379777 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.379483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.194:50537)
0809 18:52:09.379615 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.379699 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
W20260809 18:52:09.379984 3448 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:38275 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
W20260809 18:52:09.380184 3681 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:38275 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
I20260809 18:52:09.381198 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.380933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.195:39437)
0809 18:52:09.381042 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.381156 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":47,"threads_started":1}
W20260809 18:52:09.381345 3906 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:38275 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
I20260809 18:52:09.883766 4009 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 019912aa2162420e90731caaf172ec6b)
I20260809 18:52:09.883893 4009 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } }
I20260809 18:52:09.884111 4009 leader_election.cc:288] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 019912aa2162420e90731caaf172ec6b (127.23.218.254:38275), 1e4676ef914b45dd963ce1983aa14032 (127.23.218.252:43931)
I20260809 18:52:09.884356 4012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 019912aa2162420e90731caaf172ec6b)
I20260809 18:52:09.884418 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.884198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55608)
0809 18:52:09.884309 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.884376 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":42,"threads_started":1}
W20260809 18:52:09.884521 2829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
I20260809 18:52:09.884521 4012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } }
I20260809 18:52:09.884979 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.884866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55614)
0809 18:52:09.884926 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.884951 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 18:52:09.885051 2829 leader_election.cc:334] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 019912aa2162420e90731caaf172ec6b (127.23.218.254:38275): Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
I20260809 18:52:09.885269 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.884976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55620)
0809 18:52:09.885164 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.885206 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":44,"threads_started":1}
W20260809 18:52:09.885368 3123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
I20260809 18:52:09.885203 4012 leader_election.cc:288] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 019912aa2162420e90731caaf172ec6b (127.23.218.254:38275), f4fa72c6ae4e4a18b02bb7ba0dd3deec (127.23.218.253:39781)
I20260809 18:52:09.885848 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.885743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55626)
0809 18:52:09.885792 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.885822 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 18:52:09.885990 3123 leader_election.cc:334] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 019912aa2162420e90731caaf172ec6b (127.23.218.254:38275): Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
I20260809 18:52:09.887512 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.884433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.23.218.252:43931)
0809 18:52:09.884531 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:09.884532 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:09.884539 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:09.884561 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:09.884562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:09.884582 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:09.884640 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:09.884811 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.885526 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.886844 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.887205 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.887256 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.887268 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:09.887414 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 18:52:09.887452 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":43,"threads_started":1}
I20260809 18:52:09.887531 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.884269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43931 (local address 127.0.0.1:35770)
0809 18:52:09.884381 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.884390 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:09.884440 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:09.884671 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:09.884672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:09.884675 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:09.884733 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.884736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.885628 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.885629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.886756 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.886760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.887281 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.887282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.887345 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.887350 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:52:09.887394 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 18:52:09.887474 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":43,"thread_start_us":53,"threads_started":1}
I20260809 18:52:09.887754 3154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1e4676ef914b45dd963ce1983aa14032" is_pre_election: true
I20260809 18:52:09.887841 3154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fa72c6ae4e4a18b02bb7ba0dd3deec in term 1.
I20260809 18:52:09.887995 2828 leader_election.cc:302] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e4676ef914b45dd963ce1983aa14032, f4fa72c6ae4e4a18b02bb7ba0dd3deec; no voters: 019912aa2162420e90731caaf172ec6b
I20260809 18:52:09.888134 4009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 18:52:09.888177 4009 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Starting leader election (detected failure of leader 019912aa2162420e90731caaf172ec6b)
I20260809 18:52:09.888242 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.884998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:39781 (local address 127.0.0.1:41146)
0809 18:52:09.885232 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.885241 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:09.885291 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:09.885499 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:09.885500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:09.885502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:09.885551 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.885552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.886323 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.886324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.887233 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.887236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.887963 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.887964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.888037 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.888041 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:09.888098 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 18:52:09.888142 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":45,"threads_started":1}
I20260809 18:52:09.888204 4009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 1 FOLLOWER]: Advancing to term 2
I20260809 18:52:09.888330 4020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.885124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.23.218.253:39781)
0809 18:52:09.885267 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:09.885269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:09.885278 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:09.885378 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:09.885379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:09.885390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:09.885451 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:09.885625 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.886245 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.887327 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.887850 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.887919 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.887927 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:09.888135 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 18:52:09.888182 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260809 18:52:09.888543 2873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1e4676ef914b45dd963ce1983aa14032" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" is_pre_election: true
I20260809 18:52:09.889065 4009 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } }
I20260809 18:52:09.889184 2873 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1e4676ef914b45dd963ce1983aa14032 in current term 2: Already voted for candidate f4fa72c6ae4e4a18b02bb7ba0dd3deec in this term.
I20260809 18:52:09.889194 4009 leader_election.cc:288] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [CANDIDATE]: Term 2 election: Requested vote from peers 019912aa2162420e90731caaf172ec6b (127.23.218.254:38275), 1e4676ef914b45dd963ce1983aa14032 (127.23.218.252:43931)
I20260809 18:52:09.889357 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.889263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55630)
0809 18:52:09.889300 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.889337 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 18:52:09.889355 3154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1e4676ef914b45dd963ce1983aa14032"
I20260809 18:52:09.889415 3154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 1 FOLLOWER]: Advancing to term 2
I20260809 18:52:09.889681 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.889577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55646)
0809 18:52:09.889621 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.889665 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 18:52:09.889742 2829 leader_election.cc:334] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 019912aa2162420e90731caaf172ec6b (127.23.218.254:38275): Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
I20260809 18:52:09.889997 3154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fa72c6ae4e4a18b02bb7ba0dd3deec in term 2.
I20260809 18:52:09.890129 3124 leader_election.cc:302] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e4676ef914b45dd963ce1983aa14032; no voters: 019912aa2162420e90731caaf172ec6b, f4fa72c6ae4e4a18b02bb7ba0dd3deec
I20260809 18:52:09.890156 2828 leader_election.cc:302] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1e4676ef914b45dd963ce1983aa14032, f4fa72c6ae4e4a18b02bb7ba0dd3deec; no voters: 019912aa2162420e90731caaf172ec6b
I20260809 18:52:09.890230 4012 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:52:09.890244 4009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 2 FOLLOWER]: Leader election won for term 2
I20260809 18:52:09.890386 4009 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [term 2 LEADER]: Becoming Leader. State: Replica: f4fa72c6ae4e4a18b02bb7ba0dd3deec, State: Running, Role: LEADER
I20260809 18:52:09.890460 4009 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } }
I20260809 18:52:09.890807 4022 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4fa72c6ae4e4a18b02bb7ba0dd3deec. Latest consensus state: current_term: 2 leader_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:09.890898 4022 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:09.891077 4027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:09.891440 4027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:09.891505 4027 catalog_manager.cc:1259] Loaded cluster ID: 6cd7b2d124fd44a5817b7d4d2d228574
I20260809 18:52:09.891534 4027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:09.891695 4027 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:09.891767 4027 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Loaded TSK: 0
I20260809 18:52:09.891899 4027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:52:09.935061 2854 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:52:09.936604 2854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:09.937641 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.937520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55668)
0809 18:52:09.937580 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.937613 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 18:52:09.937644 3154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [term 2 FOLLOWER]: Refusing update from remote peer f4fa72c6ae4e4a18b02bb7ba0dd3deec: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 18:52:09.937880 4022 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 18:52:09.937989 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.937896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.0.0.1:55670)
0809 18:52:09.937940 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.937965 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 18:52:09.938061 2829 consensus_peers.cc:487] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec -> Peer 019912aa2162420e90731caaf172ec6b (127.23.218.254:38275): Couldn't send request to peer 019912aa2162420e90731caaf172ec6b. Status: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 18:52:09.938500 4012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4fa72c6ae4e4a18b02bb7ba0dd3deec. Latest consensus state: current_term: 2 leader_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:09.938578 4012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:52:09.938997 4009 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:09.939083 4009 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:09.939340 4012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "019912aa2162420e90731caaf172ec6b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38275 } } peers { permanent_uuid: "f4fa72c6ae4e4a18b02bb7ba0dd3deec" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 39781 } attrs { promote: false } } peers { permanent_uuid: "1e4676ef914b45dd963ce1983aa14032" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43931 } attrs { promote: false } } }
I20260809 18:52:09.939431 4012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4676ef914b45dd963ce1983aa14032 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:52:09.944258 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.940775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:35715 (local address 127.0.0.1:39646)
0809 18:52:09.940922 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.940933 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:09.940979 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:09.941339 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:09.941340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:09.941342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:09.941375 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.941376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.942186 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.942187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.943266 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.943267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.944015 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.944016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.944074 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.944077 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:09.944124 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 18:52:09.944171 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260809 18:52:09.944308 4045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.941050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.23.218.195:35715)
0809 18:52:09.941193 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:09.941195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:09.941202 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:09.941241 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:09.941242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:09.941253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:09.941433 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:09.941468 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.942062 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.943357 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.943937 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.944019 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.944027 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:09.944157 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 18:52:09.944235 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260809 18:52:09.944406 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.940914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.23.218.194:44607)
0809 18:52:09.941012 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:09.941014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:09.941021 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:09.941046 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:09.941047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:09.941057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:09.941148 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:09.941313 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.942032 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.943357 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.943958 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.944019 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.944027 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:09.944286 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 18:52:09.944339 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":48,"threads_started":1}
I20260809 18:52:09.944505 3798 tablet_service.cc:1467] Processing CreateTablet for tablet ace4995753b046fc83d107553e784b7d (DEFAULT_TABLE table=test-table [id=dbf290e9396f414290eb8da52e2f2404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:09.944622 3798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ace4995753b046fc83d107553e784b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:09.944635 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.940849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:44607 (local address 127.0.0.1:52734)
0809 18:52:09.940927 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.940936 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:09.940974 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:09.941171 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:09.941172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:09.941174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:09.941244 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.941245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.942110 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.942111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.943249 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.943252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.944059 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.944061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.944124 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.944129 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:52:09.944173 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 18:52:09.944215 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 18:52:09.944960 4049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.940859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41941 (local address 127.0.0.1:48156)
0809 18:52:09.942128 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.942136 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:09.942259 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:09.942405 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:09.942406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:09.942408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:09.942457 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.942458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.943037 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.943038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.943874 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:09.943877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.944757 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.944758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:09.944814 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.944817 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:09.944859 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 18:52:09.944895 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1249,"mutex_wait_us":297,"thread_start_us":70,"threads_started":1}
I20260809 18:52:09.944989 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.940926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.23.218.193:41941)
0809 18:52:09.941306 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:09.941308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:09.942272 (+ 964us) server_negotiation.cc:407] Connection header received
0809 18:52:09.942303 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:09.942304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:09.942315 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:09.942402 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:09.942505 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.942962 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.943959 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:09.944640 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:09.944718 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:09.944727 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:09.944887 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 18:52:09.944928 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":62,"threads_started":1}
I20260809 18:52:09.945302 3345 tablet_service.cc:1467] Processing CreateTablet for tablet ace4995753b046fc83d107553e784b7d (DEFAULT_TABLE table=test-table [id=dbf290e9396f414290eb8da52e2f2404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:09.945422 3345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ace4995753b046fc83d107553e784b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:09.945972 4050 tablet_bootstrap.cc:492] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: Bootstrap starting.
I20260809 18:52:09.946135 3570 tablet_service.cc:1467] Processing CreateTablet for tablet ace4995753b046fc83d107553e784b7d (DEFAULT_TABLE table=test-table [id=dbf290e9396f414290eb8da52e2f2404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:09.946254 3570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ace4995753b046fc83d107553e784b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:09.946650 4053 tablet_bootstrap.cc:492] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125: Bootstrap starting.
I20260809 18:52:09.946781 4050 tablet_bootstrap.cc:654] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:09.946985 4053 tablet_bootstrap.cc:654] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:09.947603 4053 tablet_bootstrap.cc:492] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125: No bootstrap required, opened a new log
I20260809 18:52:09.947649 4053 ts_tablet_manager.cc:1397] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:09.947691 4057 tablet_bootstrap.cc:492] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f: Bootstrap starting.
I20260809 18:52:09.947755 4053 raft_consensus.cc:348] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } }
I20260809 18:52:09.947820 4053 raft_consensus.cc:374] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:09.947840 4053 raft_consensus.cc:729] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d66a13491b2641e487b7554f63b90125, State: Initialized, Role: FOLLOWER
I20260809 18:52:09.947890 4053 consensus_queue.cc:260] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } }
I20260809 18:52:09.948027 4053 ts_tablet_manager.cc:1428] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:09.948132 4050 tablet_bootstrap.cc:492] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: No bootstrap required, opened a new log
I20260809 18:52:09.948184 4050 ts_tablet_manager.cc:1397] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 18:52:09.948302 4050 raft_consensus.cc:348] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } }
I20260809 18:52:09.948375 4050 raft_consensus.cc:374] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:09.948400 4050 raft_consensus.cc:729] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01c2bc35b7ee44a8ab3386e497dae5e3, State: Initialized, Role: FOLLOWER
I20260809 18:52:09.948439 4050 consensus_queue.cc:260] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } }
I20260809 18:52:09.948576 4050 ts_tablet_manager.cc:1428] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:09.948796 3450 heartbeater.cc:503] Master 127.23.218.253:39781 was elected leader, sending a full tablet report...
I20260809 18:52:09.949116 4057 tablet_bootstrap.cc:654] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:09.949486 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.948728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.195:41523)
0809 18:52:09.949415 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.949453 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":75,"threads_started":1}
I20260809 18:52:09.949664 4057 tablet_bootstrap.cc:492] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f: No bootstrap required, opened a new log
I20260809 18:52:09.949713 4057 ts_tablet_manager.cc:1397] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:52:09.949860 4057 raft_consensus.cc:348] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } }
I20260809 18:52:09.949920 4057 raft_consensus.cc:374] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:09.949934 4057 raft_consensus.cc:729] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea463fa92324405a9cad02790dd58f8f, State: Initialized, Role: FOLLOWER
I20260809 18:52:09.949965 4057 consensus_queue.cc:260] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } }
I20260809 18:52:09.950090 4057 ts_tablet_manager.cc:1428] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:09.950449 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.948259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.193:50047)
0809 18:52:09.950380 (+ 2121us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.950414 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2100,"thread_start_us":75,"threads_started":1}
I20260809 18:52:09.950619 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.950253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.194:59951)
0809 18:52:09.950552 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.950585 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":76,"threads_started":1}
I20260809 18:52:09.950807 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:09.950639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.193:45551)
0809 18:52:09.950755 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:09.950780 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
W20260809 18:52:09.979269 3912 tablet.cc:2369] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:52:10.180328 3452 tablet.cc:2369] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:52:10.191838 4063 raft_consensus.cc:482] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:10.191923 4063 raft_consensus.cc:504] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } }
I20260809 18:52:10.192297 4063 leader_election.cc:288] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d66a13491b2641e487b7554f63b90125 (127.23.218.193:41941), ea463fa92324405a9cad02790dd58f8f (127.23.218.194:44607)
I20260809 18:52:10.195207 4074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.192461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:44607 (local address 127.23.218.195:40319)
0809 18:52:10.192651 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.192661 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:10.192725 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:10.192846 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:10.192846 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:10.192848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:10.192920 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:10.192923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.193565 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.193566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:10.194461 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:10.194464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.194977 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.194978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:10.195042 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:10.195046 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:10.195099 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 18:52:10.195129 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":53,"thread_start_us":114,"threads_started":1}
I20260809 18:52:10.195228 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.192547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:40319 (local address 127.23.218.194:44607)
0809 18:52:10.192612 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:10.192613 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:10.192734 (+ 121us) server_negotiation.cc:407] Connection header received
0809 18:52:10.192767 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:10.192768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:10.192779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:10.192832 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:10.192991 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.193485 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.194542 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.194896 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.194952 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:10.194960 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:10.195122 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 18:52:10.195161 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:52:10.195299 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.192452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41941 (local address 127.23.218.195:40893)
0809 18:52:10.192594 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.192605 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:10.192782 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:10.192922 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:10.192923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:10.192925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:10.192993 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:10.192995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.193602 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.193603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:10.194473 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:10.194475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.195053 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.195054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:10.195123 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:10.195128 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:52:10.195181 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 18:52:10.195217 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":25}
I20260809 18:52:10.195309 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.192605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:40893 (local address 127.23.218.193:41941)
0809 18:52:10.192652 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:10.192654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:10.192785 (+ 131us) server_negotiation.cc:407] Connection header received
0809 18:52:10.192839 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:10.192840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:10.192849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:10.192901 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:10.193059 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.193511 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.194581 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.194941 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.195013 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:10.195023 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:10.195209 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 18:52:10.195252 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:52:10.195420 3604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace4995753b046fc83d107553e784b7d" candidate_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea463fa92324405a9cad02790dd58f8f" is_pre_election: true
I20260809 18:52:10.195503 3604 raft_consensus.cc:2454] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01c2bc35b7ee44a8ab3386e497dae5e3 in term 0.
I20260809 18:52:10.195542 3375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace4995753b046fc83d107553e784b7d" candidate_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d66a13491b2641e487b7554f63b90125" is_pre_election: true
I20260809 18:52:10.195668 3375 raft_consensus.cc:2454] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01c2bc35b7ee44a8ab3386e497dae5e3 in term 0.
I20260809 18:52:10.195863 3713 leader_election.cc:302] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01c2bc35b7ee44a8ab3386e497dae5e3, d66a13491b2641e487b7554f63b90125; no voters:
I20260809 18:52:10.196010 4063 raft_consensus.cc:2781] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:10.196053 4063 raft_consensus.cc:482] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:10.196079 4063 raft_consensus.cc:3037] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:10.196733 4063 raft_consensus.cc:504] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } }
I20260809 18:52:10.196851 4063 leader_election.cc:288] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [CANDIDATE]: Term 1 election: Requested vote from peers d66a13491b2641e487b7554f63b90125 (127.23.218.193:41941), ea463fa92324405a9cad02790dd58f8f (127.23.218.194:44607)
I20260809 18:52:10.196986 3604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace4995753b046fc83d107553e784b7d" candidate_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea463fa92324405a9cad02790dd58f8f"
I20260809 18:52:10.196993 3375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace4995753b046fc83d107553e784b7d" candidate_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d66a13491b2641e487b7554f63b90125"
I20260809 18:52:10.197055 3604 raft_consensus.cc:3037] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:10.197057 3375 raft_consensus.cc:3037] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:10.197621 3604 raft_consensus.cc:2454] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01c2bc35b7ee44a8ab3386e497dae5e3 in term 1.
I20260809 18:52:10.197645 3375 raft_consensus.cc:2454] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01c2bc35b7ee44a8ab3386e497dae5e3 in term 1.
I20260809 18:52:10.197795 3714 leader_election.cc:302] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01c2bc35b7ee44a8ab3386e497dae5e3, ea463fa92324405a9cad02790dd58f8f; no voters:
I20260809 18:52:10.197893 4063 raft_consensus.cc:2781] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:10.198046 4063 raft_consensus.cc:686] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 01c2bc35b7ee44a8ab3386e497dae5e3, State: Running, Role: LEADER
I20260809 18:52:10.198114 4063 consensus_queue.cc:237] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } }
I20260809 18:52:10.198173 3910 heartbeater.cc:503] Master 127.23.218.253:39781 was elected leader, sending a full tablet report...
I20260809 18:52:10.198311 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.198235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.195:59511)
0809 18:52:10.198263 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.198290 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
I20260809 18:52:10.198899 2854 catalog_manager.cc:5495] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01c2bc35b7ee44a8ab3386e497dae5e3 (127.23.218.195). New cstate: current_term: 1 leader_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01c2bc35b7ee44a8ab3386e497dae5e3" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 35715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 } health_report { overall_health: UNKNOWN } } }
I20260809 18:52:10.225483 4045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.223288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.23.218.195:35715)
0809 18:52:10.223363 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:10.223365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:10.223372 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:10.223403 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:10.223404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:10.223420 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:10.223478 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:10.223605 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.224044 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.224774 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:10.224954 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:10.225008 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:10.225163 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:10.225259 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:10.225320 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:10.225400 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 18:52:10.225419 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 18:52:10.226521 3604 raft_consensus.cc:1261] T ace4995753b046fc83d107553e784b7d P ea463fa92324405a9cad02790dd58f8f [term 1 FOLLOWER]: Refusing update from remote peer 01c2bc35b7ee44a8ab3386e497dae5e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:10.226610 3375 raft_consensus.cc:1261] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [term 1 FOLLOWER]: Refusing update from remote peer 01c2bc35b7ee44a8ab3386e497dae5e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:10.226770 4075 consensus_queue.cc:1035] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea463fa92324405a9cad02790dd58f8f" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:10.226895 4063 consensus_queue.cc:1035] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d66a13491b2641e487b7554f63b90125" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 41941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:10.227285 3682 heartbeater.cc:503] Master 127.23.218.253:39781 was elected leader, sending a full tablet report...
I20260809 18:52:10.227581 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.227477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.194:36819)
0809 18:52:10.227519 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.227546 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 18:52:10.228142 4078 mvcc.cc:205] Tried to move back new op lower bound from 7316691067805999104 to 7316691067691991040. Current Snapshot: MvccSnapshot[applied={T|T < 7316691067805999104}]
I20260809 18:52:10.228638 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.228429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.194:43673)
0809 18:52:10.228584 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.228614 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260809 18:52:10.228826 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.228327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.195:52449)
0809 18:52:10.228773 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.228801 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431}
I20260809 18:52:10.229187 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.229096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.193:45659)
0809 18:52:10.229135 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.229158 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 18:52:10.230849 4065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.230761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.195:44151)
0809 18:52:10.230800 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.230823 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 18:52:10.231287 4062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.231159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38275 (local address 127.23.218.193:39579)
0809 18:52:10.231238 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.231263 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 18:52:10.231420 2852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:10.231501 2852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:10.232667 2852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4fa72c6ae4e4a18b02bb7ba0dd3deec: Sending DeleteTablet for 3 replicas of tablet ace4995753b046fc83d107553e784b7d
I20260809 18:52:10.232949 3345 tablet_service.cc:1514] Processing DeleteTablet for tablet ace4995753b046fc83d107553e784b7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:10 UTC) from {username='oryx', principal='oryx/127.23.218.253@KRBTEST.COM'} at 127.0.0.1:48156
I20260809 18:52:10.233042 3570 tablet_service.cc:1514] Processing DeleteTablet for tablet ace4995753b046fc83d107553e784b7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:10 UTC) from {username='oryx', principal='oryx/127.23.218.253@KRBTEST.COM'} at 127.0.0.1:52734
I20260809 18:52:10.233103 3798 tablet_service.cc:1514] Processing DeleteTablet for tablet ace4995753b046fc83d107553e784b7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:10 UTC) from {username='oryx', principal='oryx/127.23.218.253@KRBTEST.COM'} at 127.0.0.1:39646
I20260809 18:52:10.233134 4094 tablet_replica.cc:330] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125: stopping tablet replica
I20260809 18:52:10.233183 4094 raft_consensus.cc:2229] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:10.233222 4094 raft_consensus.cc:2258] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:10.233255 4096 tablet_replica.cc:330] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: stopping tablet replica
I20260809 18:52:10.233390 4096 raft_consensus.cc:2229] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:52:10.233459 4096 raft_consensus.cc:2258] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:10.233494 4094 ts_tablet_manager.cc:1905] T ace4995753b046fc83d107553e784b7d P d66a13491b2641e487b7554f63b90125: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:10.233626 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 3238
I20260809 18:52:10.233755 4096 ts_tablet_manager.cc:1905] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:10.235558 4096 ts_tablet_manager.cc:1918] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:10.235610 4096 log.cc:1196] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/ace4995753b046fc83d107553e784b7d
I20260809 18:52:10.235852 4096 ts_tablet_manager.cc:1939] T ace4995753b046fc83d107553e784b7d P 01c2bc35b7ee44a8ab3386e497dae5e3: Deleting consensus metadata
I20260809 18:52:10.236174 2827 catalog_manager.cc:4841] TS 01c2bc35b7ee44a8ab3386e497dae5e3 (127.23.218.195:35715): tablet ace4995753b046fc83d107553e784b7d (table test-table [id=dbf290e9396f414290eb8da52e2f2404]) successfully deleted
W20260809 18:52:10.243247 4049 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:10.243125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:41941 (local address 127.0.0.1:48164)
0809 18:52:10.243166 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.243196 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:41941: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 18:52:10.243367 2827 catalog_manager.cc:4584] TS d66a13491b2641e487b7554f63b90125 (127.23.218.193:41941): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ace4995753b046fc83d107553e784b7d: Network error: Client connection negotiation failed: client connection to 127.23.218.193:41941: connect: Connection reset by peer (error 104)
I20260809 18:52:10.243468 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 3460
I20260809 18:52:10.276856 4049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:10.276748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:44607 (local address 127.0.0.1:52736)
0809 18:52:10.276791 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:10.276820 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.194:44607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 18:52:10.277043 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 3693
I20260809 18:52:10.287681 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 2635
I20260809 18:52:10.297375 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 2958
2026-08-09T18:52:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4883 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:52:10 dist-test-slave-6zbq krb5kdc[4099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:52:10 dist-test-slave-6zbq krb5kdc[4099](info): set up 2 sockets
Aug 09 18:52:10 dist-test-slave-6zbq krb5kdc[4099](info): commencing operation
krb5kdc: starting...
W20260809 18:52:11.444299 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:52:11 dist-test-slave-6zbq krb5kdc[4099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:52:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.254.keytab.
Entry for principal oryx/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.254.keytab.
I20260809 18:52:11.553177 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:41167
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.254.keytab
--principal=oryx/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:41167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:11.631795 4113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:11.631995 4113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:11.632030 4113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:11.633153 4113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:11.633193 4113 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 18:52:11.633209 4113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:11.633226 4113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:11.633244 4113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:11.633258 4113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:11.634757 4113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:41167
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.254.keytab
--principal=oryx/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:41167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:11.634981 4113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:11.635195 4113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:11.639676 4122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:11.639748 4113 server_base.cc:1034] running on GCE node
W20260809 18:52:11.639673 4120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:11.639669 4126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:11.640066 4113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:11.640322 4113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:11.641466 4113 hybrid_clock.cc:648] HybridClock initialized: now 1786301531641450 us; error 27 us; skew 500 ppm
Aug 09 18:52:11 dist-test-slave-6zbq krb5kdc[4099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301531, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:11.643893 4113 init.cc:377] Logged in from keytab as oryx/127.23.218.254@KRBTEST.COM (short username oryx)
I20260809 18:52:11.644246 4113 webserver.cc:460] Webserver started at http://127.23.218.254:43751/ using document root <none> and password file <none>
I20260809 18:52:11.644407 4113 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:11.644444 4113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:11.644531 4113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:11.645453 4113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "97053079adc7491fa529722ee712cdbd"
format_stamp: "Formatted at 2026-08-09 18:52:11 on dist-test-slave-6zbq"
server_key: "1b33b7f212be5cec2c710468c313a426"
server_key_iv: "e47dc3634e849a869a6fcfc9b9ca41a0"
server_key_version: "encryptionkey@0"
I20260809 18:52:11.645732 4113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "97053079adc7491fa529722ee712cdbd"
format_stamp: "Formatted at 2026-08-09 18:52:11 on dist-test-slave-6zbq"
server_key: "1b33b7f212be5cec2c710468c313a426"
server_key_iv: "e47dc3634e849a869a6fcfc9b9ca41a0"
server_key_version: "encryptionkey@0"
I20260809 18:52:11.646956 4113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:11.647567 4137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:11.647722 4113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:11.647790 4113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "97053079adc7491fa529722ee712cdbd"
format_stamp: "Formatted at 2026-08-09 18:52:11 on dist-test-slave-6zbq"
server_key: "1b33b7f212be5cec2c710468c313a426"
server_key_iv: "e47dc3634e849a869a6fcfc9b9ca41a0"
server_key_version: "encryptionkey@0"
I20260809 18:52:11.647845 4113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:11.655493 4113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:11.656993 4113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:11.657097 4113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:11.661650 4113 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:41167
I20260809 18:52:11.661684 4231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:41167 every 8 connection(s)
I20260809 18:52:11.661965 4113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:11.662580 4233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:11.664598 4233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd: Bootstrap starting.
I20260809 18:52:11.665135 4233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:11.665402 4233 log.cc:824] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:11.665985 4233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd: No bootstrap required, opened a new log
I20260809 18:52:11.667125 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 4113
I20260809 18:52:11.667225 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:11.667416 24427 external_mini_cluster.cc:1428] Setting key 31199dd8389476c6065b2e42e9398e0c
I20260809 18:52:11.668275 4233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97053079adc7491fa529722ee712cdbd" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41167 } }
I20260809 18:52:11.668401 4233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:11.668431 4233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97053079adc7491fa529722ee712cdbd, State: Initialized, Role: FOLLOWER
I20260809 18:52:11.668548 4233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97053079adc7491fa529722ee712cdbd" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41167 } }
I20260809 18:52:11.668619 4233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:11.668650 4233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:11.668697 4233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:11.669157 4233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97053079adc7491fa529722ee712cdbd" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41167 } }
I20260809 18:52:11.669255 4233 leader_election.cc:302] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97053079adc7491fa529722ee712cdbd; no voters:
I20260809 18:52:11.669399 4233 leader_election.cc:288] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:11.669447 4242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:11.669556 4242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [term 1 LEADER]: Becoming Leader. State: Replica: 97053079adc7491fa529722ee712cdbd, State: Running, Role: LEADER
I20260809 18:52:11.669636 4233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:11.669631 4242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97053079adc7491fa529722ee712cdbd" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41167 } }
I20260809 18:52:11.670255 4244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97053079adc7491fa529722ee712cdbd. Latest consensus state: current_term: 1 leader_uuid: "97053079adc7491fa529722ee712cdbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97053079adc7491fa529722ee712cdbd" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41167 } } }
I20260809 18:52:11.670229 4243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97053079adc7491fa529722ee712cdbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97053079adc7491fa529722ee712cdbd" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41167 } } }
I20260809 18:52:11.670418 4243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:11.670372 4244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:11.670629 4253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:11.671294 4253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:11.672806 4253 catalog_manager.cc:1347] Generated new cluster ID: 7aca3754e3404ef084a5a20b36991d5d
I20260809 18:52:11.672863 4253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:52:11 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:11.678475 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:11.667916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50408 (local address 127.23.218.254:41167)
0809 18:52:11.668095 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:11.668103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:11.668127 (+ 24us) server_negotiation.cc:407] Connection header received
0809 18:52:11.668238 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:11.668243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:11.668283 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:11.668394 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:11.669067 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.670272 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.671375 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.671883 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.671979 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:11.674620 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:11.674624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:11.674624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:11.674635 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:11.676918 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.677297 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.677300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.677301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.677418 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.677611 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.677612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.677613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.677914 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:11.678067 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:11.678135 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 18:52:11.678269 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260809 18:52:11.682307 4253 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:11.682796 4253 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:11.690433 4253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd: Generated new TSK 0
I20260809 18:52:11.690567 4253 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab.
Entry for principal oryx/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab.
I20260809 18:52:11.717396 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:41167
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab
--principal=oryx/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:11.796401 4268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:11.796578 4268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:11.796599 4268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:11.797698 4268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:11.797740 4268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:11.797760 4268 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 18:52:11.797789 4268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:11.799293 4268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab
--principal=oryx/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.218.254:41167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:11.799503 4268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:11.799724 4268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:11.804322 4279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:11.804391 4277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:11.804479 4268 server_base.cc:1034] running on GCE node
W20260809 18:52:11.804405 4276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:11.804725 4268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:11.804955 4268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:11.806093 4268 hybrid_clock.cc:648] HybridClock initialized: now 1786301531806075 us; error 32 us; skew 500 ppm
Aug 09 18:52:11 dist-test-slave-6zbq krb5kdc[4099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301531, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:11.808516 4268 init.cc:377] Logged in from keytab as oryx/127.23.218.193@KRBTEST.COM (short username oryx)
I20260809 18:52:11.808915 4268 webserver.cc:460] Webserver started at http://127.23.218.193:45735/ using document root <none> and password file <none>
I20260809 18:52:11.809100 4268 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:11.809137 4268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:11.809224 4268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:11.810148 4268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "cb6df2e2f66849e2a92907caba674e49"
format_stamp: "Formatted at 2026-08-09 18:52:11 on dist-test-slave-6zbq"
server_key: "a4ec5767b0830e72d1fffb015113b893"
server_key_iv: "ef6b3af1fc7f142592ad47b8689cc77a"
server_key_version: "encryptionkey@0"
I20260809 18:52:11.810468 4268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "cb6df2e2f66849e2a92907caba674e49"
format_stamp: "Formatted at 2026-08-09 18:52:11 on dist-test-slave-6zbq"
server_key: "a4ec5767b0830e72d1fffb015113b893"
server_key_iv: "ef6b3af1fc7f142592ad47b8689cc77a"
server_key_version: "encryptionkey@0"
I20260809 18:52:11.811610 4268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:52:11.812275 4288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:11.812438 4268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:11.812513 4268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "cb6df2e2f66849e2a92907caba674e49"
format_stamp: "Formatted at 2026-08-09 18:52:11 on dist-test-slave-6zbq"
server_key: "a4ec5767b0830e72d1fffb015113b893"
server_key_iv: "ef6b3af1fc7f142592ad47b8689cc77a"
server_key_version: "encryptionkey@0"
I20260809 18:52:11.812580 4268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:11.825292 4268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:11.827293 4268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:11.827416 4268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:11.827888 4268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:11.827929 4268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:11.827950 4268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:11.827966 4268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 18:52:11 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301531, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:11.834445 4268 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:35635
I20260809 18:52:11.834474 4467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:35635 every 8 connection(s)
I20260809 18:52:11.834824 4268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:52:11.836982 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:11.828239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:47853 (local address 127.23.218.254:41167)
0809 18:52:11.828290 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:11.828292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:11.828457 (+ 165us) server_negotiation.cc:407] Connection header received
0809 18:52:11.828902 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:11.828904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:11.828919 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:11.828968 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:11.829626 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.830399 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.831827 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.832086 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.832150 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:11.834366 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:11.834367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:11.834368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:11.834375 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:11.835714 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.836074 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.836075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.836076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.836135 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.836387 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.836388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.836389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.836496 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:11.836552 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:11.836811 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 18:52:11.836896 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:52:11.837105 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:11.828212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41167 (local address 127.23.218.193:47853)
0809 18:52:11.828350 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:11.828383 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:11.828805 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:11.829031 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:11.829034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:11.829327 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:11.829533 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:11.829538 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.830485 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.830486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:11.831729 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:11.831733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.832192 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.832194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:11.832338 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:11.832767 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:11.832776 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:11.834266 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:11.835795 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:11.835802 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:11.835804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:11.836001 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:11.836195 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:11.836196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:11.836196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:11.836324 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:11.836558 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:11.836564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:11.836687 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 18:52:11.836772 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 18:52:11.836913 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260809 18:52:11.837567 4468 heartbeater.cc:347] Connected to a master server at 127.23.218.254:41167
I20260809 18:52:11.837642 4468 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:11.837788 4468 heartbeater.cc:511] Master 127.23.218.254:41167 requested a full tablet report, sending...
I20260809 18:52:11.838308 4155 ts_manager.cc:194] Registered new tserver with Master: cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635)
I20260809 18:52:11.839001 4155 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:47853
I20260809 18:52:11.841867 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 4268
I20260809 18:52:11.841974 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:52:11.842152 24427 external_mini_cluster.cc:1428] Setting key 8ec67d4d9aa92458fbd5d12b7b3992b9
I20260809 18:52:11.844110 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:11.839187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:37685 (local address 127.23.218.254:41167)
0809 18:52:11.839236 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:11.839238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:11.839245 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:11.839264 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:11.839265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:11.839274 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:11.839333 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:11.839679 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.840107 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.840751 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.840917 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.840960 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:11.841721 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:11.841722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:11.841722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:11.841728 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:11.843196 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.843523 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.843523 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.843523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.843568 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.843732 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.843732 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.843733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.843824 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:11.843868 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:11.844003 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 18:52:11.844057 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 18:52:11.844161 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:11.838427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41167 (local address 127.23.218.193:37685)
0809 18:52:11.838546 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:11.838556 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:11.838605 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:11.839356 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:11.839357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:11.839551 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:11.839608 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:11.839610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.840188 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.840189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:11.840689 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:11.840690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.840971 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.840971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:11.841013 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:11.841389 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:11.841393 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:11.841643 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:11.843272 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:11.843273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:11.843273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:11.843452 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:11.843624 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:11.843624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:11.843625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:11.843682 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:11.843877 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:11.843878 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:11.843922 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 18:52:11.843988 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 18:52:11.844072 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab.
Entry for principal oryx/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab.
I20260809 18:52:11.864512 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:41167
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab
--principal=oryx/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:11.941623 4485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:11.941809 4485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:11.941831 4485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:11.942960 4485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:11.942997 4485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:11.943017 4485 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 18:52:11.943037 4485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:11.944530 4485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab
--principal=oryx/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.218.254:41167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:11.944705 4485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:11.944876 4485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:11.949359 4497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:11.949385 4493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:11.949385 4494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:11.949602 4485 server_base.cc:1034] running on GCE node
I20260809 18:52:11.949817 4485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:11.950028 4485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:11.951174 4485 hybrid_clock.cc:648] HybridClock initialized: now 1786301531951152 us; error 36 us; skew 500 ppm
Aug 09 18:52:11 dist-test-slave-6zbq krb5kdc[4099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301531, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:11.953547 4485 init.cc:377] Logged in from keytab as oryx/127.23.218.194@KRBTEST.COM (short username oryx)
I20260809 18:52:11.953931 4485 webserver.cc:460] Webserver started at http://127.23.218.194:33259/ using document root <none> and password file <none>
I20260809 18:52:11.954114 4485 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:11.954172 4485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:11.954300 4485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:11.955310 4485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "94d5c08ca7254b87b7ee2d2d5c63e271"
format_stamp: "Formatted at 2026-08-09 18:52:11 on dist-test-slave-6zbq"
server_key: "0da37deb6427234357d5a416565e173e"
server_key_iv: "77362695faaf49e19ec99a7fa588016d"
server_key_version: "encryptionkey@0"
I20260809 18:52:11.955648 4485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "94d5c08ca7254b87b7ee2d2d5c63e271"
format_stamp: "Formatted at 2026-08-09 18:52:11 on dist-test-slave-6zbq"
server_key: "0da37deb6427234357d5a416565e173e"
server_key_iv: "77362695faaf49e19ec99a7fa588016d"
server_key_version: "encryptionkey@0"
I20260809 18:52:11.956923 4485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:52:11.957579 4506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:11.957746 4485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:11.957827 4485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "94d5c08ca7254b87b7ee2d2d5c63e271"
format_stamp: "Formatted at 2026-08-09 18:52:11 on dist-test-slave-6zbq"
server_key: "0da37deb6427234357d5a416565e173e"
server_key_iv: "77362695faaf49e19ec99a7fa588016d"
server_key_version: "encryptionkey@0"
I20260809 18:52:11.957890 4485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:11.973448 4485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:11.975407 4485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:11.975507 4485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:11.975895 4485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:11.975930 4485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:11.975948 4485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:11.975962 4485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 18:52:11 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301531, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:11.982813 4485 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:39399
I20260809 18:52:11.982975 4692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:39399 every 8 connection(s)
I20260809 18:52:11.983177 4485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:52:11.985057 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:11.976340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:47375 (local address 127.23.218.254:41167)
0809 18:52:11.976391 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:11.976393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:11.976584 (+ 191us) server_negotiation.cc:407] Connection header received
0809 18:52:11.977040 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:11.977041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:11.977055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:11.977114 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:11.977953 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.978717 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.980119 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.980338 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.980390 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:11.982673 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:11.982675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:11.982675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:11.982684 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:11.984076 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.984398 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.984399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.984399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.984443 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.984599 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.984600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.984600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.984688 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:11.984731 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:11.984941 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 18:52:11.984997 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 18:52:11.985179 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:11.976322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41167 (local address 127.23.218.194:47375)
0809 18:52:11.976474 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:11.976507 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:11.976942 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:11.977227 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:11.977232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:11.977563 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:11.977849 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:11.977856 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.978796 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.978798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:11.980027 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:11.980031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.980422 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.980423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:11.980551 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:11.981045 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:11.981054 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:11.982571 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:11.984142 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:11.984145 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:11.984146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:11.984336 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:11.984479 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:11.984479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:11.984480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:11.984556 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:11.984735 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:11.984738 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:11.984842 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 18:52:11.984916 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 18:52:11.985022 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260809 18:52:11.985590 4693 heartbeater.cc:347] Connected to a master server at 127.23.218.254:41167
I20260809 18:52:11.985673 4693 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:11.985853 4693 heartbeater.cc:511] Master 127.23.218.254:41167 requested a full tablet report, sending...
I20260809 18:52:11.986260 4156 ts_manager.cc:194] Registered new tserver with Master: 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399)
I20260809 18:52:11.986682 4156 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:47375
I20260809 18:52:11.988467 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 4485
I20260809 18:52:11.988577 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:52:11.988780 24427 external_mini_cluster.cc:1428] Setting key 278957c14e0d09697dff8e3c7c743d14
I20260809 18:52:11.991568 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:11.986599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:53351 (local address 127.23.218.254:41167)
0809 18:52:11.986643 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:11.986645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:11.986740 (+ 95us) server_negotiation.cc:407] Connection header received
0809 18:52:11.986810 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:11.986811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:11.986821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:11.986868 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:11.987152 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.987583 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.988212 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.988393 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.988432 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:11.989212 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:11.989213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:11.989213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:11.989219 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:11.990500 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.990833 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.990834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.990834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.990876 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:11.991079 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:11.991080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:11.991080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:11.991211 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:11.991267 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:11.991421 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 18:52:11.991483 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:52:11.991578 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:11.986522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41167 (local address 127.23.218.194:53351)
0809 18:52:11.986674 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:11.986683 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:11.986731 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:11.986880 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:11.986881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:11.987028 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:11.987078 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:11.987080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.987671 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.987672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:11.988149 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:11.988151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:11.988461 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:11.988462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:11.988528 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:11.988859 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:11.988864 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:11.989144 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:11.990571 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:11.990572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:11.990573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:11.990765 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:11.990947 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:11.990948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:11.990949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:11.991010 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:11.991281 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:11.991282 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:11.991327 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 18:52:11.991386 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 18:52:11.991478 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for oryx/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab.
Entry for principal oryx/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab.
I20260809 18:52:12.012601 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:41167
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab
--principal=oryx/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:12.101148 4706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:12.101330 4706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:12.101353 4706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:12.102491 4706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:12.102527 4706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:12.102547 4706 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 18:52:12.102568 4706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:12.104058 4706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab
--principal=oryx/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.218.254:41167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:12.104229 4706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:12.104393 4706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:12.109228 4716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:12.109299 4712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:12.109232 4706 server_base.cc:1034] running on GCE node
W20260809 18:52:12.109272 4714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:12.109625 4706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:12.109854 4706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:12.110993 4706 hybrid_clock.cc:648] HybridClock initialized: now 1786301532110977 us; error 27 us; skew 500 ppm
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301532, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:12.113328 4706 init.cc:377] Logged in from keytab as oryx/127.23.218.195@KRBTEST.COM (short username oryx)
I20260809 18:52:12.113713 4706 webserver.cc:460] Webserver started at http://127.23.218.195:44141/ using document root <none> and password file <none>
I20260809 18:52:12.113904 4706 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:12.113951 4706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:12.114066 4706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:12.115305 4706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "52332febb242438dbf08e727c4383e99"
format_stamp: "Formatted at 2026-08-09 18:52:12 on dist-test-slave-6zbq"
server_key: "355e851e114eca56d175902acfbe5e4d"
server_key_iv: "e55772be3b02c3b1b7ce657c5664b8c9"
server_key_version: "encryptionkey@0"
I20260809 18:52:12.115667 4706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "52332febb242438dbf08e727c4383e99"
format_stamp: "Formatted at 2026-08-09 18:52:12 on dist-test-slave-6zbq"
server_key: "355e851e114eca56d175902acfbe5e4d"
server_key_iv: "e55772be3b02c3b1b7ce657c5664b8c9"
server_key_version: "encryptionkey@0"
I20260809 18:52:12.117057 4706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:12.117833 4727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:12.118016 4706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:12.118098 4706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "52332febb242438dbf08e727c4383e99"
format_stamp: "Formatted at 2026-08-09 18:52:12 on dist-test-slave-6zbq"
server_key: "355e851e114eca56d175902acfbe5e4d"
server_key_iv: "e55772be3b02c3b1b7ce657c5664b8c9"
server_key_version: "encryptionkey@0"
I20260809 18:52:12.118171 4706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:12.134213 4706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:12.135927 4706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:12.136040 4706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:12.136694 4706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:12.136734 4706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:12.136752 4706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:12.136762 4706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301532, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:12.146932 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.137051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:36653 (local address 127.23.218.254:41167)
0809 18:52:12.137101 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.137103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.137486 (+ 383us) server_negotiation.cc:407] Connection header received
0809 18:52:12.138044 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.138046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.138060 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.138131 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:12.138803 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.139549 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.140916 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.141162 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.141225 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.144151 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:12.144152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:12.144153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:12.144168 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:12.145568 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.145956 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.145958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.145958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.146018 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.146265 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.146266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.146266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.146430 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:12.146492 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.146756 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.146844 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:52:12.147138 4740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.137042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41167 (local address 127.23.218.195:36653)
0809 18:52:12.137341 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.137378 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.137935 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.138185 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.138187 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.138462 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:12.138708 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.138716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.139643 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.139644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.140820 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.140824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.141247 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.141248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.141509 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.142084 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:12.142094 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:12.144057 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:12.145655 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:12.145662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:12.145664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:12.145877 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:12.146081 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:12.146082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:12.146082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:12.146208 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:12.146493 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:12.146500 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:12.146626 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.146713 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.146863 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":182,"threads_started":1}
I20260809 18:52:12.150439 4706 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:41321
I20260809 18:52:12.150640 4925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:41321 every 8 connection(s)
I20260809 18:52:12.150870 4706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:52:12.151557 4927 heartbeater.cc:347] Connected to a master server at 127.23.218.254:41167
I20260809 18:52:12.152148 4927 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:12.152328 4927 heartbeater.cc:511] Master 127.23.218.254:41167 requested a full tablet report, sending...
I20260809 18:52:12.152719 4156 ts_manager.cc:194] Registered new tserver with Master: 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321)
I20260809 18:52:12.153190 4156 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:36653
I20260809 18:52:12.155365 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.148955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:60903 (local address 127.23.218.254:41167)
0809 18:52:12.148992 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.148994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.149202 (+ 208us) server_negotiation.cc:407] Connection header received
0809 18:52:12.149228 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.149229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.149239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.149299 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:12.150173 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.150875 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.151878 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.152105 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.152174 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.153133 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:12.153134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:12.153134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:12.153140 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:12.154505 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.154809 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.154810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.154810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.154855 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.154970 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.154970 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.154971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.155050 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:12.155090 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.155252 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.155304 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:52:12.155390 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.148916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41167 (local address 127.23.218.195:60903)
0809 18:52:12.149126 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.149136 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.149192 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.149331 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.149332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.149666 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:12.150073 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.150076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.151126 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.151127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.151808 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.151811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.152186 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.152187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.152254 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.152683 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:12.152688 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:12.153046 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:12.154576 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:12.154577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:12.154577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:12.154741 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:12.154893 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:12.154893 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:12.154894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:12.154938 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:12.155084 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:12.155084 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:12.155171 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.155226 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.155295 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":95,"threads_started":1}
I20260809 18:52:12.157253 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 4706
I20260809 18:52:12.157348 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:52:12.157522 24427 external_mini_cluster.cc:1428] Setting key 1f74af343b64e07cfb5fba00e5947467
I20260809 18:52:12.158277 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301532, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 18:52:12.179971 4241 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:12.175023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.23.218.254:41167)
0809 18:52:12.175115 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.175117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.175151 (+ 34us) server_negotiation.cc:407] Connection header received
0809 18:52:12.175172 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.175173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.175182 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.175229 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:12.175768 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.176258 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.176905 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.177072 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.177125 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.179919 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50412: BlockingRecv error: recv got EOF from 127.0.0.1:50412 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301532, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:12.186532 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.180699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.23.218.254:41167)
0809 18:52:12.180771 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.180772 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.180780 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:12.180842 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.180843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.180850 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.180894 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:12.181291 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.181715 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.182313 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.182539 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.182588 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.184415 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:12.184417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:12.184417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:12.184423 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:12.185636 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.185960 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.185960 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.185961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.186009 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.186146 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.186147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.186147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.186230 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:12.186294 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.186407 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.186473 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 18:52:12.187850 4156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:52:12.189333 4156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:12.195349 4953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.191502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:35635 (local address 127.0.0.1:41404)
0809 18:52:12.191875 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.191883 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.192014 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.192171 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.192171 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.192173 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:12.192219 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.192220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.193264 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.193266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.194042 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.194044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.195065 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.195066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.195142 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.195156 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.195234 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.195286 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":42,"threads_started":1}
I20260809 18:52:12.195461 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.191331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41321 (local address 127.0.0.1:35484)
0809 18:52:12.191473 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.191507 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.191563 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.191827 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.191830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.191833 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:12.191891 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.191893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.193109 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.193110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.194508 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.194510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.195257 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.195258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.195323 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.195326 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.195370 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.195404 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":28,"thread_start_us":50,"threads_started":1}
I20260809 18:52:12.195582 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.191331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:39399 (local address 127.0.0.1:41454)
0809 18:52:12.191645 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.191654 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.191713 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.191906 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.191907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.191909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:12.191962 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.191964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.193411 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.193412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.194479 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.194481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.195376 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.195377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.195444 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.195449 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.195479 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.195518 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":1,"thread_start_us":45,"threads_started":1}
I20260809 18:52:12.195709 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.191418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35484 (local address 127.23.218.195:41321)
0809 18:52:12.191594 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.191599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.191609 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:52:12.191648 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.191650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.191666 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.191748 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:12.191970 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.193015 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.194587 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.195086 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.195168 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.195224 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.195553 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.195598 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260809 18:52:12.195869 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.191692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41404 (local address 127.23.218.193:35635)
0809 18:52:12.191854 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.191859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.192021 (+ 162us) server_negotiation.cc:407] Connection header received
0809 18:52:12.192071 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.192073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.192088 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.192170 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:12.192297 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.193173 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.194137 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.194968 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.195088 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.195144 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.195278 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.195323 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260809 18:52:12.196188 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.191486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41454 (local address 127.23.218.194:39399)
0809 18:52:12.191643 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.191646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.191707 (+ 61us) server_negotiation.cc:407] Connection header received
0809 18:52:12.191771 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.191773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.191787 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.191907 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:12.192241 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.193327 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.194571 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.195268 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.195963 (+ 695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.196033 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.196059 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.196098 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":63,"threads_started":1}
I20260809 18:52:12.196591 4363 tablet_service.cc:1467] Processing CreateTablet for tablet 1192f10ac702401b86a6f1dc7535d3b9 (DEFAULT_TABLE table=test-table [id=2067722c886446c0a7cf74adb2d9e173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.196651 4804 tablet_service.cc:1467] Processing CreateTablet for tablet 1192f10ac702401b86a6f1dc7535d3b9 (DEFAULT_TABLE table=test-table [id=2067722c886446c0a7cf74adb2d9e173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.196983 4363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1192f10ac702401b86a6f1dc7535d3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.196998 4804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1192f10ac702401b86a6f1dc7535d3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.199016 4593 tablet_service.cc:1467] Processing CreateTablet for tablet 1192f10ac702401b86a6f1dc7535d3b9 (DEFAULT_TABLE table=test-table [id=2067722c886446c0a7cf74adb2d9e173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.199419 4593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1192f10ac702401b86a6f1dc7535d3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.200712 4959 tablet_bootstrap.cc:492] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: Bootstrap starting.
I20260809 18:52:12.201537 4961 tablet_bootstrap.cc:492] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: Bootstrap starting.
I20260809 18:52:12.201673 4959 tablet_bootstrap.cc:654] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.202289 4959 log.cc:824] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:12.202677 4961 tablet_bootstrap.cc:654] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.203013 4959 tablet_bootstrap.cc:492] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: No bootstrap required, opened a new log
I20260809 18:52:12.203071 4959 ts_tablet_manager.cc:1397] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:52:12.203444 4961 log.cc:824] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:12.203450 4964 tablet_bootstrap.cc:492] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: Bootstrap starting.
I20260809 18:52:12.204180 4961 tablet_bootstrap.cc:492] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: No bootstrap required, opened a new log
I20260809 18:52:12.204233 4961 ts_tablet_manager.cc:1397] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 18:52:12.204622 4964 tablet_bootstrap.cc:654] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.205334 4964 log.cc:824] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:12.206147 4964 tablet_bootstrap.cc:492] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: No bootstrap required, opened a new log
I20260809 18:52:12.206213 4964 ts_tablet_manager.cc:1397] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 18:52:12.206139 4959 raft_consensus.cc:348] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } }
I20260809 18:52:12.206314 4959 raft_consensus.cc:374] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.206398 4959 raft_consensus.cc:729] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6df2e2f66849e2a92907caba674e49, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.206532 4959 consensus_queue.cc:260] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } }
I20260809 18:52:12.206611 4961 raft_consensus.cc:348] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } }
I20260809 18:52:12.206768 4961 raft_consensus.cc:374] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.206841 4959 ts_tablet_manager.cc:1428] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 18:52:12.206820 4961 raft_consensus.cc:729] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52332febb242438dbf08e727c4383e99, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.207015 4468 heartbeater.cc:503] Master 127.23.218.254:41167 was elected leader, sending a full tablet report...
I20260809 18:52:12.207108 4961 consensus_queue.cc:260] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } }
I20260809 18:52:12.207388 4961 ts_tablet_manager.cc:1428] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 18:52:12.207425 4927 heartbeater.cc:503] Master 127.23.218.254:41167 was elected leader, sending a full tablet report...
I20260809 18:52:12.209283 4964 raft_consensus.cc:348] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } }
I20260809 18:52:12.209416 4964 raft_consensus.cc:374] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.209436 4964 raft_consensus.cc:729] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d5c08ca7254b87b7ee2d2d5c63e271, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.209524 4964 consensus_queue.cc:260] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } }
I20260809 18:52:12.209779 4964 ts_tablet_manager.cc:1428] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 18:52:12.209851 4693 heartbeater.cc:503] Master 127.23.218.254:41167 was elected leader, sending a full tablet report...
W20260809 18:52:12.233749 4697 tablet.cc:2369] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:52:12.335784 4472 tablet.cc:2369] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:52:12.396327 4970 raft_consensus.cc:482] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:12.396458 4970 raft_consensus.cc:504] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } }
I20260809 18:52:12.396787 4970 leader_election.cc:288] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321), cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635)
I20260809 18:52:12.400653 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.397080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:34659 (local address 127.23.218.195:41321)
0809 18:52:12.397133 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.397135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.397145 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:52:12.397178 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.397179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.397188 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.397266 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:12.397438 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.398234 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.399718 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.400258 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.400335 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.400344 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.400558 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.400601 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 18:52:12.400728 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.396982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41321 (local address 127.23.218.194:34659)
0809 18:52:12.397039 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.397051 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.397097 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.397295 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.397295 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.397297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:12.397367 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.397369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.398316 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.398316 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.399626 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.399630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.400342 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.400342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.400484 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.400489 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.400547 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.400594 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":45}
I20260809 18:52:12.401000 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.397186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:40237 (local address 127.23.218.193:35635)
0809 18:52:12.397226 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.397227 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.397286 (+ 59us) server_negotiation.cc:407] Connection header received
0809 18:52:12.397332 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.397333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.397340 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.397382 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:12.397519 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.398241 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.399717 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.400267 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.400335 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.400344 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.400894 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.400937 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:52:12.401008 4971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.397091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:35635 (local address 127.23.218.194:40237)
0809 18:52:12.397223 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.397231 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.397281 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.397399 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.397400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.397401 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:12.397457 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.397458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.398313 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.398313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.399626 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.399630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.400342 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.400342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.400823 (+ 481us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.400827 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.400884 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.400921 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":49,"thread_start_us":91,"threads_started":1}
I20260809 18:52:12.401126 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1192f10ac702401b86a6f1dc7535d3b9" candidate_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52332febb242438dbf08e727c4383e99" is_pre_election: true
I20260809 18:52:12.401283 4831 raft_consensus.cc:2454] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94d5c08ca7254b87b7ee2d2d5c63e271 in term 0.
W20260809 18:52:12.401505 4928 tablet.cc:2369] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:52:12.401546 4509 leader_election.cc:302] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52332febb242438dbf08e727c4383e99, 94d5c08ca7254b87b7ee2d2d5c63e271; no voters:
I20260809 18:52:12.401664 4970 raft_consensus.cc:2781] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:12.401726 4970 raft_consensus.cc:482] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:12.401749 4970 raft_consensus.cc:3037] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.401785 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1192f10ac702401b86a6f1dc7535d3b9" candidate_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6df2e2f66849e2a92907caba674e49" is_pre_election: true
I20260809 18:52:12.401921 4401 raft_consensus.cc:2454] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94d5c08ca7254b87b7ee2d2d5c63e271 in term 0.
I20260809 18:52:12.402601 4970 raft_consensus.cc:504] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } }
I20260809 18:52:12.402709 4970 leader_election.cc:288] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [CANDIDATE]: Term 1 election: Requested vote from peers 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321), cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635)
I20260809 18:52:12.402856 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1192f10ac702401b86a6f1dc7535d3b9" candidate_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52332febb242438dbf08e727c4383e99"
I20260809 18:52:12.402901 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1192f10ac702401b86a6f1dc7535d3b9" candidate_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6df2e2f66849e2a92907caba674e49"
I20260809 18:52:12.402935 4831 raft_consensus.cc:3037] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.402969 4401 raft_consensus.cc:3037] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.403870 4401 raft_consensus.cc:2454] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94d5c08ca7254b87b7ee2d2d5c63e271 in term 1.
I20260809 18:52:12.404057 4509 leader_election.cc:302] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94d5c08ca7254b87b7ee2d2d5c63e271, cb6df2e2f66849e2a92907caba674e49; no voters:
I20260809 18:52:12.404196 4831 raft_consensus.cc:2454] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94d5c08ca7254b87b7ee2d2d5c63e271 in term 1.
I20260809 18:52:12.404199 4970 raft_consensus.cc:2781] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:12.404369 4970 raft_consensus.cc:686] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 LEADER]: Becoming Leader. State: Replica: 94d5c08ca7254b87b7ee2d2d5c63e271, State: Running, Role: LEADER
I20260809 18:52:12.404467 4970 consensus_queue.cc:237] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } }
I20260809 18:52:12.405326 4156 catalog_manager.cc:5495] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194). New cstate: current_term: 1 leader_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } health_report { overall_health: UNKNOWN } } }
I20260809 18:52:12.425040 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.422762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41470 (local address 127.23.218.194:39399)
0809 18:52:12.422807 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.422809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.422826 (+ 17us) server_negotiation.cc:407] Connection header received
0809 18:52:12.422866 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.422867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.422888 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.422950 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:12.423089 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.423552 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.424298 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.424495 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.424551 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.424723 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:12.424832 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:12.424904 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.424949 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.424968 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 18:52:12.426972 4401 raft_consensus.cc:1261] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Refusing update from remote peer 94d5c08ca7254b87b7ee2d2d5c63e271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:12.427006 4831 raft_consensus.cc:1261] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Refusing update from remote peer 94d5c08ca7254b87b7ee2d2d5c63e271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:12.427276 4970 consensus_queue.cc:1035] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:12.427433 4973 consensus_queue.cc:1035] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:12.431699 4982 mvcc.cc:205] Tried to move back new op lower bound from 7316691076817965056 to 7316691076729135104. Current Snapshot: MvccSnapshot[applied={T|T < 7316691076817965056 or (T in {7316691076817965056})}]
I20260809 18:52:12.431983 4155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:12.432072 4155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:12.432821 4155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd: Sending DeleteTablet for 3 replicas of tablet 1192f10ac702401b86a6f1dc7535d3b9
I20260809 18:52:12.433159 4804 tablet_service.cc:1514] Processing DeleteTablet for tablet 1192f10ac702401b86a6f1dc7535d3b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:12 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:35484
I20260809 18:52:12.433247 4363 tablet_service.cc:1514] Processing DeleteTablet for tablet 1192f10ac702401b86a6f1dc7535d3b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:12 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41404
I20260809 18:52:12.433491 4593 tablet_service.cc:1514] Processing DeleteTablet for tablet 1192f10ac702401b86a6f1dc7535d3b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:12 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41454
I20260809 18:52:12.433635 4155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 18:52:12.433722 4997 tablet_replica.cc:330] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: stopping tablet replica
I20260809 18:52:12.433810 4997 raft_consensus.cc:2229] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 18:52:12.433789 4155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:12.433902 4997 raft_consensus.cc:2258] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:12.433735 4998 tablet_replica.cc:330] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: stopping tablet replica
I20260809 18:52:12.434098 4998 raft_consensus.cc:2229] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:12.434244 4998 raft_consensus.cc:2258] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:12.434379 4997 ts_tablet_manager.cc:1905] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:12.434794 4998 ts_tablet_manager.cc:1905] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:12.435213 4363 tablet_service.cc:1467] Processing CreateTablet for tablet 7724727765ac43409966b4d1f49b16fa (DEFAULT_TABLE table=test-table [id=eb3f2c5d4ed841a58cd6bb91fd14676a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.435302 4593 tablet_service.cc:1467] Processing CreateTablet for tablet 7724727765ac43409966b4d1f49b16fa (DEFAULT_TABLE table=test-table [id=eb3f2c5d4ed841a58cd6bb91fd14676a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.435339 4363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7724727765ac43409966b4d1f49b16fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.435388 4593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7724727765ac43409966b4d1f49b16fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.436363 4997 ts_tablet_manager.cc:1918] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:12.436426 4997 log.cc:1196] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/1192f10ac702401b86a6f1dc7535d3b9
I20260809 18:52:12.436676 4959 tablet_bootstrap.cc:492] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: Bootstrap starting.
I20260809 18:52:12.436941 4964 tablet_bootstrap.cc:492] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: Bootstrap starting.
I20260809 18:52:12.437076 4804 tablet_service.cc:1467] Processing CreateTablet for tablet 7724727765ac43409966b4d1f49b16fa (DEFAULT_TABLE table=test-table [id=eb3f2c5d4ed841a58cd6bb91fd14676a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.437183 4804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7724727765ac43409966b4d1f49b16fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.437369 4964 tablet_bootstrap.cc:654] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.437523 4959 tablet_bootstrap.cc:654] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.438426 4997 ts_tablet_manager.cc:1939] T 1192f10ac702401b86a6f1dc7535d3b9 P cb6df2e2f66849e2a92907caba674e49: Deleting consensus metadata
I20260809 18:52:12.438540 4961 tablet_bootstrap.cc:492] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: Bootstrap starting.
I20260809 18:52:12.438869 4140 catalog_manager.cc:4841] TS cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635): tablet 1192f10ac702401b86a6f1dc7535d3b9 (table test-table [id=2067722c886446c0a7cf74adb2d9e173]) successfully deleted
I20260809 18:52:12.439378 4961 tablet_bootstrap.cc:654] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.439466 4998 ts_tablet_manager.cc:1918] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:12.439584 4998 log.cc:1196] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/1192f10ac702401b86a6f1dc7535d3b9
I20260809 18:52:12.439903 4998 ts_tablet_manager.cc:1939] T 1192f10ac702401b86a6f1dc7535d3b9 P 52332febb242438dbf08e727c4383e99: Deleting consensus metadata
I20260809 18:52:12.440363 4140 catalog_manager.cc:4841] TS 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321): tablet 1192f10ac702401b86a6f1dc7535d3b9 (table test-table [id=2067722c886446c0a7cf74adb2d9e173]) successfully deleted
I20260809 18:52:12.440388 4961 tablet_bootstrap.cc:492] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: No bootstrap required, opened a new log
I20260809 18:52:12.440430 4961 ts_tablet_manager.cc:1397] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 18:52:12.440729 4961 raft_consensus.cc:348] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.440809 4961 raft_consensus.cc:374] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.440793 4999 tablet_replica.cc:330] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: stopping tablet replica
I20260809 18:52:12.440829 4961 raft_consensus.cc:729] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52332febb242438dbf08e727c4383e99, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.440831 4964 tablet_bootstrap.cc:492] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: No bootstrap required, opened a new log
I20260809 18:52:12.440869 4964 ts_tablet_manager.cc:1397] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260809 18:52:12.440876 4961 consensus_queue.cc:260] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.440919 4999 raft_consensus.cc:2229] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:52:12.440961 4961 ts_tablet_manager.cc:1428] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:12.441051 4999 raft_consensus.cc:2258] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:12.441237 4964 raft_consensus.cc:348] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.441295 4964 raft_consensus.cc:374] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.441313 4964 raft_consensus.cc:729] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d5c08ca7254b87b7ee2d2d5c63e271, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.441360 4964 consensus_queue.cc:260] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.441444 4964 ts_tablet_manager.cc:1428] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 18:52:12.441509 4999 ts_tablet_manager.cc:1905] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:12.442727 4959 tablet_bootstrap.cc:492] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: No bootstrap required, opened a new log
I20260809 18:52:12.442771 4959 ts_tablet_manager.cc:1397] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260809 18:52:12.443050 4959 raft_consensus.cc:348] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.443109 4959 raft_consensus.cc:374] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.443126 4959 raft_consensus.cc:729] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6df2e2f66849e2a92907caba674e49, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.443162 4959 consensus_queue.cc:260] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.443245 4959 ts_tablet_manager.cc:1428] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:12.443600 4999 ts_tablet_manager.cc:1918] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:12.443668 4999 log.cc:1196] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/1192f10ac702401b86a6f1dc7535d3b9
I20260809 18:52:12.443922 4999 ts_tablet_manager.cc:1939] T 1192f10ac702401b86a6f1dc7535d3b9 P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting consensus metadata
I20260809 18:52:12.444305 4142 catalog_manager.cc:4841] TS 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399): tablet 1192f10ac702401b86a6f1dc7535d3b9 (table test-table [id=2067722c886446c0a7cf74adb2d9e173]) successfully deleted
I20260809 18:52:12.487028 4967 raft_consensus.cc:482] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:12.487124 4967 raft_consensus.cc:504] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.487462 4967 leader_election.cc:288] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399), 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321)
I20260809 18:52:12.491217 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.487803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:33915 (local address 127.23.218.195:41321)
0809 18:52:12.487847 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.487850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.487883 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:52:12.487910 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.487911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.487921 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.487969 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:12.488287 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.489033 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.490415 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.490761 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.490824 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.490833 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.491088 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.491136 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:52:12.491225 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.487646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41321 (local address 127.23.218.193:33915)
0809 18:52:12.487769 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.487782 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.487842 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.488014 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.488015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.488017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:12.488219 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.488222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.489121 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.489122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.490275 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.490278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.490904 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.490905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.491013 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.491019 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.491077 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.491125 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":5,"thread_start_us":56,"threads_started":1}
I20260809 18:52:12.491294 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.487707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:50165 (local address 127.23.218.194:39399)
0809 18:52:12.487930 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.487932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.488067 (+ 135us) server_negotiation.cc:407] Connection header received
0809 18:52:12.488130 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.488132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.488152 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.488211 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:12.488422 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.488878 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.490394 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.490937 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.491007 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.491015 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.491195 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.491240 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
I20260809 18:52:12.491509 5004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.487713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:39399 (local address 127.23.218.193:50165)
0809 18:52:12.488006 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.488015 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.488069 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.488237 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.488238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.488266 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:12.488344 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.488345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.488960 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.488961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.490274 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.490278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.491026 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.491027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.491108 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.491111 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.491376 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.491422 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":1,"thread_start_us":166,"threads_started":1}
I20260809 18:52:12.491854 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7724727765ac43409966b4d1f49b16fa" candidate_uuid: "cb6df2e2f66849e2a92907caba674e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" is_pre_election: true
I20260809 18:52:12.491909 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7724727765ac43409966b4d1f49b16fa" candidate_uuid: "cb6df2e2f66849e2a92907caba674e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52332febb242438dbf08e727c4383e99" is_pre_election: true
I20260809 18:52:12.491995 4831 raft_consensus.cc:2454] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb6df2e2f66849e2a92907caba674e49 in term 0.
I20260809 18:52:12.492005 4625 raft_consensus.cc:2454] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb6df2e2f66849e2a92907caba674e49 in term 0.
I20260809 18:52:12.492208 4294 leader_election.cc:302] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94d5c08ca7254b87b7ee2d2d5c63e271, cb6df2e2f66849e2a92907caba674e49; no voters:
I20260809 18:52:12.492334 4967 raft_consensus.cc:2781] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:12.492372 4967 raft_consensus.cc:482] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:12.492389 4967 raft_consensus.cc:3037] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.493369 4967 raft_consensus.cc:504] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.493489 4967 leader_election.cc:288] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [CANDIDATE]: Term 1 election: Requested vote from peers 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399), 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321)
I20260809 18:52:12.493638 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7724727765ac43409966b4d1f49b16fa" candidate_uuid: "cb6df2e2f66849e2a92907caba674e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271"
I20260809 18:52:12.493712 4625 raft_consensus.cc:3037] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.493762 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7724727765ac43409966b4d1f49b16fa" candidate_uuid: "cb6df2e2f66849e2a92907caba674e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52332febb242438dbf08e727c4383e99"
I20260809 18:52:12.493832 4831 raft_consensus.cc:3037] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.494238 4625 raft_consensus.cc:2454] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb6df2e2f66849e2a92907caba674e49 in term 1.
I20260809 18:52:12.494453 4294 leader_election.cc:302] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94d5c08ca7254b87b7ee2d2d5c63e271, cb6df2e2f66849e2a92907caba674e49; no voters:
I20260809 18:52:12.494532 4967 raft_consensus.cc:2781] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:12.494609 4831 raft_consensus.cc:2454] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb6df2e2f66849e2a92907caba674e49 in term 1.
I20260809 18:52:12.494653 4967 raft_consensus.cc:686] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 1 LEADER]: Becoming Leader. State: Replica: cb6df2e2f66849e2a92907caba674e49, State: Running, Role: LEADER
I20260809 18:52:12.494720 4967 consensus_queue.cc:237] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.495556 4156 catalog_manager.cc:5495] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb6df2e2f66849e2a92907caba674e49 (127.23.218.193). New cstate: current_term: 1 leader_uuid: "cb6df2e2f66849e2a92907caba674e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } health_report { overall_health: UNKNOWN } } }
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301531, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.254@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:12.522742 5014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.514975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41167 (local address 127.0.0.1:50432)
0809 18:52:12.515144 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.515152 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.515215 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.515325 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.515326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.515664 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:12.515711 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.515714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.516414 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.516415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.516931 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.516932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.517248 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.517249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.517296 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.517900 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:12.517905 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:12.519519 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:12.521292 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:12.521300 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:12.521303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:12.521562 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:12.521868 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:12.521869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:12.521869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:12.521967 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:12.522317 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:12.522325 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:12.522411 (+ 86us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.522488 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.522605 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260809 18:52:12.523123 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.515027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50432 (local address 127.23.218.254:41167)
0809 18:52:12.515076 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.515078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.515237 (+ 159us) server_negotiation.cc:407] Connection header received
0809 18:52:12.515261 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.515261 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.515271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.515316 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:12.515788 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.516337 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.517008 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.517181 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.517234 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.519619 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:12.519620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:12.519621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:12.519628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:12.521204 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.521682 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.521684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.521684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.521748 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.522078 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.522079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.522080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.522256 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:12.522329 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.522523 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.522663 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":23}
I20260809 18:52:12.529055 4156 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:50432:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 18:52:12.529255 4156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:12.530853 4804 tablet_service.cc:1467] Processing CreateTablet for tablet 8e0f4dae9ede4925a04f71dd54452c23 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b9d6bcf0ac4451fad6b34d2ac450802]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 18:52:12.530869 4593 tablet_service.cc:1467] Processing CreateTablet for tablet 8e0f4dae9ede4925a04f71dd54452c23 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b9d6bcf0ac4451fad6b34d2ac450802]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 18:52:12.531021 4593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e0f4dae9ede4925a04f71dd54452c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.530987 4363 tablet_service.cc:1467] Processing CreateTablet for tablet 8e0f4dae9ede4925a04f71dd54452c23 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b9d6bcf0ac4451fad6b34d2ac450802]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 18:52:12.531026 4804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e0f4dae9ede4925a04f71dd54452c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.531163 4363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e0f4dae9ede4925a04f71dd54452c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.532670 4961 tablet_bootstrap.cc:492] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: Bootstrap starting.
I20260809 18:52:12.532719 4964 tablet_bootstrap.cc:492] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: Bootstrap starting.
I20260809 18:52:12.533455 4961 tablet_bootstrap.cc:654] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.533535 4959 tablet_bootstrap.cc:492] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: Bootstrap starting.
I20260809 18:52:12.533564 4964 tablet_bootstrap.cc:654] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.534591 4959 tablet_bootstrap.cc:654] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.534600 4961 tablet_bootstrap.cc:492] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: No bootstrap required, opened a new log
I20260809 18:52:12.534643 4961 ts_tablet_manager.cc:1397] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 18:52:12.535017 4964 tablet_bootstrap.cc:492] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: No bootstrap required, opened a new log
I20260809 18:52:12.535064 4964 ts_tablet_manager.cc:1397] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 18:52:12.535041 4961 raft_consensus.cc:348] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.535120 4961 raft_consensus.cc:374] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.535138 4961 raft_consensus.cc:729] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52332febb242438dbf08e727c4383e99, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.535182 4961 consensus_queue.cc:260] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.535199 4964 raft_consensus.cc:348] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.535266 4964 raft_consensus.cc:374] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.535264 4961 ts_tablet_manager.cc:1428] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 18:52:12.535286 4964 raft_consensus.cc:729] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d5c08ca7254b87b7ee2d2d5c63e271, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.535327 4964 consensus_queue.cc:260] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.535421 4964 ts_tablet_manager.cc:1428] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:12.535452 4973 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.535553 4973 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:52:12.535621 4968 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.535727 4968 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:52:12.536418 4959 tablet_bootstrap.cc:492] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: No bootstrap required, opened a new log
I20260809 18:52:12.536463 4959 ts_tablet_manager.cc:1397] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 18:52:12.536583 4959 raft_consensus.cc:348] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.536649 4959 raft_consensus.cc:374] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.536664 4959 raft_consensus.cc:729] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6df2e2f66849e2a92907caba674e49, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.536693 4959 consensus_queue.cc:260] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.536779 4959 ts_tablet_manager.cc:1428] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:12.536896 4967 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.536954 4967 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:52:12.576819 4968 raft_consensus.cc:482] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:12.576912 4968 raft_consensus.cc:504] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.577204 4968 leader_election.cc:288] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635), 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399)
I20260809 18:52:12.579890 4740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.577350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:35635 (local address 127.23.218.195:60379)
0809 18:52:12.577402 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.577415 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.577468 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.577588 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.577589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.577590 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:12.577666 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.577668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.578284 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.578284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.579074 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.579076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.579640 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.579641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.579735 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.579738 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.579784 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.579820 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 18:52:12.579896 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.577442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:60379 (local address 127.23.218.193:35635)
0809 18:52:12.577492 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.577494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.577503 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:12.577529 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.577530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.577539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.577586 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:12.577736 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.578207 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.579185 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.579563 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.579634 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.579643 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.579807 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.579841 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 18:52:12.580152 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e0f4dae9ede4925a04f71dd54452c23" candidate_uuid: "52332febb242438dbf08e727c4383e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6df2e2f66849e2a92907caba674e49" is_pre_election: true
I20260809 18:52:12.580243 4401 raft_consensus.cc:2454] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52332febb242438dbf08e727c4383e99 in term 0.
I20260809 18:52:12.580250 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.577463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:32781 (local address 127.23.218.194:39399)
0809 18:52:12.577513 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.577515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.577626 (+ 111us) server_negotiation.cc:407] Connection header received
0809 18:52:12.577672 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.577673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.577681 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.577725 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:12.577868 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.578304 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.579506 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.579842 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.579905 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.579914 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.580129 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.580183 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 18:52:12.580451 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.577351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:39399 (local address 127.23.218.195:32781)
0809 18:52:12.577554 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.577563 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.577618 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.577738 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.577739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.577740 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:12.577810 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.577811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.578391 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.578392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.579414 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.579417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.579935 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.579936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.580012 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.580016 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.580328 (+ 312us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.580371 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":16,"thread_start_us":126,"threads_started":1}
I20260809 18:52:12.580452 4730 leader_election.cc:302] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52332febb242438dbf08e727c4383e99, cb6df2e2f66849e2a92907caba674e49; no voters:
I20260809 18:52:12.580663 4968 raft_consensus.cc:2781] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:12.580704 4968 raft_consensus.cc:482] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:12.580722 4968 raft_consensus.cc:3037] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.580686 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e0f4dae9ede4925a04f71dd54452c23" candidate_uuid: "52332febb242438dbf08e727c4383e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" is_pre_election: true
I20260809 18:52:12.580762 4625 raft_consensus.cc:2454] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52332febb242438dbf08e727c4383e99 in term 0.
I20260809 18:52:12.581444 4968 raft_consensus.cc:504] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.581568 4968 leader_election.cc:288] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [CANDIDATE]: Term 1 election: Requested vote from peers cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635), 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399)
I20260809 18:52:12.581739 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e0f4dae9ede4925a04f71dd54452c23" candidate_uuid: "52332febb242438dbf08e727c4383e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271"
I20260809 18:52:12.581745 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e0f4dae9ede4925a04f71dd54452c23" candidate_uuid: "52332febb242438dbf08e727c4383e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6df2e2f66849e2a92907caba674e49"
I20260809 18:52:12.581806 4625 raft_consensus.cc:3037] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.581813 4401 raft_consensus.cc:3037] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.582420 4401 raft_consensus.cc:2454] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52332febb242438dbf08e727c4383e99 in term 1.
I20260809 18:52:12.582461 4625 raft_consensus.cc:2454] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52332febb242438dbf08e727c4383e99 in term 1.
I20260809 18:52:12.582576 4730 leader_election.cc:302] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52332febb242438dbf08e727c4383e99, cb6df2e2f66849e2a92907caba674e49; no voters:
I20260809 18:52:12.582683 4968 raft_consensus.cc:2781] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:12.582836 4968 raft_consensus.cc:686] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [term 1 LEADER]: Becoming Leader. State: Replica: 52332febb242438dbf08e727c4383e99, State: Running, Role: LEADER
I20260809 18:52:12.582902 4968 consensus_queue.cc:237] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.583280 5024 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: TxnStatusTablet state changed. Reason: New leader 52332febb242438dbf08e727c4383e99. Latest consensus state: current_term: 1 leader_uuid: "52332febb242438dbf08e727c4383e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.583379 5024 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: This TxnStatusTablet replica's current role is: LEADER
I20260809 18:52:12.583561 5026 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 18:52:12.583668 5026 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 18:52:12.583618 4156 catalog_manager.cc:5495] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52332febb242438dbf08e727c4383e99 (127.23.218.195). New cstate: current_term: 1 leader_uuid: "52332febb242438dbf08e727c4383e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } health_report { overall_health: HEALTHY } } }
I20260809 18:52:12.597214 5014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.594670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41321 (local address 127.0.0.1:35498)
0809 18:52:12.594746 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.594758 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.594805 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.594957 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.594957 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.594958 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:52:12.594998 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.594999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.595828 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.595829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.596369 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.596371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.596816 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.596817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.596858 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.596872 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:52:12.597096 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:52:12.597100 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.597123 (+ 23us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.597153 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 18:52:12.597210 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.594737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35498 (local address 127.23.218.195:41321)
0809 18:52:12.594785 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.594786 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.594796 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:52:12.594865 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.594865 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.594884 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.594956 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:12.595044 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.595760 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.596447 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.596752 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.596817 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.596931 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:12.597044 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:12.597108 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.597128 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.597145 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 18:52:12.598196 4401 raft_consensus.cc:1261] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Refusing update from remote peer 52332febb242438dbf08e727c4383e99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:12.598264 4625 raft_consensus.cc:1261] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Refusing update from remote peer 52332febb242438dbf08e727c4383e99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:12.598453 5024 consensus_queue.cc:1035] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:12.598570 4968 consensus_queue.cc:1035] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:12.599162 4967 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: TxnStatusTablet state changed. Reason: New leader 52332febb242438dbf08e727c4383e99. Latest consensus state: current_term: 1 leader_uuid: "52332febb242438dbf08e727c4383e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.599252 4967 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:52:12.599217 4973 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: TxnStatusTablet state changed. Reason: New leader 52332febb242438dbf08e727c4383e99. Latest consensus state: current_term: 1 leader_uuid: "52332febb242438dbf08e727c4383e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.599295 4973 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:52:12.599709 4968 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52332febb242438dbf08e727c4383e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.599789 4968 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: This TxnStatusTablet replica's current role is: LEADER
I20260809 18:52:12.600183 4983 mvcc.cc:205] Tried to move back new op lower bound from 7316691077520482304 to 7316691077459992576. Current Snapshot: MvccSnapshot[applied={T|T < 7316691077520482304 or (T in {7316691077520482304})}]
I20260809 18:52:12.600389 5024 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "52332febb242438dbf08e727c4383e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.600466 5024 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P 52332febb242438dbf08e727c4383e99: This TxnStatusTablet replica's current role is: LEADER
I20260809 18:52:12.600615 4967 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52332febb242438dbf08e727c4383e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.600710 4967 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P cb6df2e2f66849e2a92907caba674e49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:52:12.600987 4973 tablet_replica.cc:439] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "52332febb242438dbf08e727c4383e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } } }
I20260809 18:52:12.601060 4973 tablet_replica.cc:442] T 8e0f4dae9ede4925a04f71dd54452c23 P 94d5c08ca7254b87b7ee2d2d5c63e271: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 18:52:12.604415 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.601940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.23.218.193:35635)
0809 18:52:12.601981 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.601983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.601991 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:12.602053 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.602054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.602073 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.602129 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:12.602262 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.602748 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.603522 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.603831 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.603889 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.604085 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:12.604206 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:12.604269 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.604336 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.604357 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:52:12.613320 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.608968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:50765 (local address 127.23.218.195:41321)
0809 18:52:12.609058 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.609060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.609673 (+ 613us) server_negotiation.cc:407] Connection header received
0809 18:52:12.609738 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.609739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.609747 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.609815 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:12.610064 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.610942 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.612338 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.612684 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.612741 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.613014 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:12.613044 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:12.613091 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.613216 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.613236 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 18:52:12.613744 5035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.608914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41321 (local address 127.23.218.193:50765)
0809 18:52:12.609606 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.609617 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.609685 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.609847 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.609848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.609885 (+ 37us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:52:12.609969 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.609971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.611053 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.611054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.612214 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.612217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.612793 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.612793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.612878 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.612914 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:52:12.613127 (+ 213us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:52:12.613128 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.613554 (+ 426us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.613622 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":84,"threads_started":1}
I20260809 18:52:12.620272 4831 raft_consensus.cc:1261] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Refusing update from remote peer cb6df2e2f66849e2a92907caba674e49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:12.620301 4625 raft_consensus.cc:1261] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Refusing update from remote peer cb6df2e2f66849e2a92907caba674e49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:12.620520 5005 consensus_queue.cc:1035] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:12.620625 4967 consensus_queue.cc:1035] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:12.622152 4981 mvcc.cc:205] Tried to move back new op lower bound from 7316691077610577920 to 7316691077098913792. Current Snapshot: MvccSnapshot[applied={T|T < 7316691077610577920}]
I20260809 18:52:12.629066 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.626688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:35635 (local address 127.23.218.195:39937)
0809 18:52:12.626762 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:12.626771 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:12.626816 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:12.626960 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:12.626961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:12.626963 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:52:12.627018 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.627019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.627657 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.627665 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.628215 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:12.628216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.628598 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.628599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:12.628658 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.628677 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:52:12.628912 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:52:12.628914 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:52:12.628950 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 18:52:12.628994 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 18:52:12.629295 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.626733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:39937 (local address 127.23.218.193:35635)
0809 18:52:12.626773 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.626774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.626805 (+ 31us) server_negotiation.cc:407] Connection header received
0809 18:52:12.626860 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.626861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.626870 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.626918 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:12.627085 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.627575 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.628298 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.628506 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.628564 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.628758 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:12.628790 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:12.628838 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.629217 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.629237 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:52:12.638940 4156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:12.639035 4156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:12.639611 4156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd: Sending DeleteTablet for 3 replicas of tablet 7724727765ac43409966b4d1f49b16fa
I20260809 18:52:12.639856 4363 tablet_service.cc:1514] Processing DeleteTablet for tablet 7724727765ac43409966b4d1f49b16fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:12 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41404
I20260809 18:52:12.639941 4997 tablet_replica.cc:330] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: stopping tablet replica
I20260809 18:52:12.639982 4997 raft_consensus.cc:2229] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:52:12.640056 4997 raft_consensus.cc:2258] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:12.640123 4593 tablet_service.cc:1514] Processing DeleteTablet for tablet 7724727765ac43409966b4d1f49b16fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:12 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41454
I20260809 18:52:12.640251 4999 tablet_replica.cc:330] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: stopping tablet replica
I20260809 18:52:12.640296 4999 raft_consensus.cc:2229] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:12.640326 4999 raft_consensus.cc:2258] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:12.640441 4804 tablet_service.cc:1514] Processing DeleteTablet for tablet 7724727765ac43409966b4d1f49b16fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:12 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:35484
I20260809 18:52:12.640514 4998 tablet_replica.cc:330] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: stopping tablet replica
I20260809 18:52:12.640551 4998 raft_consensus.cc:2229] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:12.640579 4998 raft_consensus.cc:2258] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:12.641052 4998 ts_tablet_manager.cc:1905] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:12.642551 4999 ts_tablet_manager.cc:1905] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:12.642655 4998 ts_tablet_manager.cc:1918] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:52:12.642719 4998 log.cc:1196] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/7724727765ac43409966b4d1f49b16fa
I20260809 18:52:12.642925 4998 ts_tablet_manager.cc:1939] T 7724727765ac43409966b4d1f49b16fa P 52332febb242438dbf08e727c4383e99: Deleting consensus metadata
I20260809 18:52:12.643229 4140 catalog_manager.cc:4841] TS 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321): tablet 7724727765ac43409966b4d1f49b16fa (table test-table [id=eb3f2c5d4ed841a58cd6bb91fd14676a]) successfully deleted
I20260809 18:52:12.644050 4999 ts_tablet_manager.cc:1918] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:52:12.644101 4999 log.cc:1196] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/7724727765ac43409966b4d1f49b16fa
I20260809 18:52:12.644289 4999 ts_tablet_manager.cc:1939] T 7724727765ac43409966b4d1f49b16fa P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting consensus metadata
I20260809 18:52:12.644734 4997 ts_tablet_manager.cc:1905] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:12.646380 4997 ts_tablet_manager.cc:1918] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:52:12.646435 4997 log.cc:1196] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/7724727765ac43409966b4d1f49b16fa
I20260809 18:52:12.646628 4997 ts_tablet_manager.cc:1939] T 7724727765ac43409966b4d1f49b16fa P cb6df2e2f66849e2a92907caba674e49: Deleting consensus metadata
I20260809 18:52:12.646955 4140 catalog_manager.cc:4841] TS cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635): tablet 7724727765ac43409966b4d1f49b16fa (table test-table [id=eb3f2c5d4ed841a58cd6bb91fd14676a]) successfully deleted
I20260809 18:52:12.647086 4142 catalog_manager.cc:4841] TS 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399): tablet 7724727765ac43409966b4d1f49b16fa (table test-table [id=eb3f2c5d4ed841a58cd6bb91fd14676a]) successfully deleted
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 18:52:12.674301 4241 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:12.667445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50440 (local address 127.23.218.254:41167)
0809 18:52:12.667489 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.667492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.667641 (+ 149us) server_negotiation.cc:407] Connection header received
0809 18:52:12.667690 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.667691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.667702 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.667766 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:12.668497 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.669268 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.670416 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.670751 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.670814 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.674240 (+ 3426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50440: BlockingRecv error: recv got EOF from 127.0.0.1:50440 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 18:52:12 dist-test-slave-6zbq krb5kdc[4099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM
I20260809 18:52:12.687160 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.675284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50442 (local address 127.23.218.254:41167)
0809 18:52:12.675321 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.675322 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.678522 (+ 3200us) server_negotiation.cc:407] Connection header received
0809 18:52:12.678636 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.678637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.678648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.678712 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:12.679297 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.680054 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.681066 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.681385 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.681452 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.684099 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:12.684100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:12.684101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:12.684108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:12.685849 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.686239 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.686240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.686241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.686303 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:12.686596 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:12.686597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:12.686597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:12.686770 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:12.686880 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.686989 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.687068 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:52:12.688966 4156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:52:12.689139 4156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:12.690709 4363 tablet_service.cc:1467] Processing CreateTablet for tablet f00a0be5af8b47858a7a80d4aaa2fa20 (DEFAULT_TABLE table=test-table [id=6af557ad0a3e402c934a581eddc5dfd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.690864 4363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f00a0be5af8b47858a7a80d4aaa2fa20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.692318 4804 tablet_service.cc:1467] Processing CreateTablet for tablet f00a0be5af8b47858a7a80d4aaa2fa20 (DEFAULT_TABLE table=test-table [id=6af557ad0a3e402c934a581eddc5dfd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.692438 4804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f00a0be5af8b47858a7a80d4aaa2fa20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.693822 4593 tablet_service.cc:1467] Processing CreateTablet for tablet f00a0be5af8b47858a7a80d4aaa2fa20 (DEFAULT_TABLE table=test-table [id=6af557ad0a3e402c934a581eddc5dfd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.693940 4593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f00a0be5af8b47858a7a80d4aaa2fa20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.694465 4961 tablet_bootstrap.cc:492] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: Bootstrap starting.
I20260809 18:52:12.694882 4961 tablet_bootstrap.cc:654] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.695187 4959 tablet_bootstrap.cc:492] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: Bootstrap starting.
I20260809 18:52:12.695322 4964 tablet_bootstrap.cc:492] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: Bootstrap starting.
I20260809 18:52:12.695511 4959 tablet_bootstrap.cc:654] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.695637 4964 tablet_bootstrap.cc:654] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.696457 4959 tablet_bootstrap.cc:492] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: No bootstrap required, opened a new log
I20260809 18:52:12.696506 4959 ts_tablet_manager.cc:1397] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:12.696556 4964 tablet_bootstrap.cc:492] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: No bootstrap required, opened a new log
I20260809 18:52:12.696600 4964 ts_tablet_manager.cc:1397] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 18:52:12.696702 4964 raft_consensus.cc:348] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.696762 4964 raft_consensus.cc:374] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.696781 4964 raft_consensus.cc:729] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d5c08ca7254b87b7ee2d2d5c63e271, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.696743 4959 raft_consensus.cc:348] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.696807 4959 raft_consensus.cc:374] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.696826 4959 raft_consensus.cc:729] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6df2e2f66849e2a92907caba674e49, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.696834 4964 consensus_queue.cc:260] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.696863 4959 consensus_queue.cc:260] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.696938 4964 ts_tablet_manager.cc:1428] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:12.696949 4959 ts_tablet_manager.cc:1428] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:12.697983 4961 tablet_bootstrap.cc:492] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: No bootstrap required, opened a new log
I20260809 18:52:12.698041 4961 ts_tablet_manager.cc:1397] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260809 18:52:12.698203 4961 raft_consensus.cc:348] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.698274 4961 raft_consensus.cc:374] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.698292 4961 raft_consensus.cc:729] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52332febb242438dbf08e727c4383e99, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.698331 4961 consensus_queue.cc:260] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.698587 4961 ts_tablet_manager.cc:1428] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 18:52:12.789613 4973 raft_consensus.cc:482] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:12.789713 4973 raft_consensus.cc:504] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.789863 4973 leader_election.cc:288] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635), 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321)
I20260809 18:52:12.790068 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00a0be5af8b47858a7a80d4aaa2fa20" candidate_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6df2e2f66849e2a92907caba674e49" is_pre_election: true
I20260809 18:52:12.790170 4401 raft_consensus.cc:2454] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94d5c08ca7254b87b7ee2d2d5c63e271 in term 0.
I20260809 18:52:12.790127 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00a0be5af8b47858a7a80d4aaa2fa20" candidate_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52332febb242438dbf08e727c4383e99" is_pre_election: true
I20260809 18:52:12.790207 4831 raft_consensus.cc:2454] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94d5c08ca7254b87b7ee2d2d5c63e271 in term 0.
I20260809 18:52:12.790318 4509 leader_election.cc:302] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94d5c08ca7254b87b7ee2d2d5c63e271, cb6df2e2f66849e2a92907caba674e49; no voters:
I20260809 18:52:12.790477 4973 raft_consensus.cc:2781] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:12.790524 4973 raft_consensus.cc:482] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:12.790544 4973 raft_consensus.cc:3037] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.791257 4973 raft_consensus.cc:504] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.791373 4973 leader_election.cc:288] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [CANDIDATE]: Term 1 election: Requested vote from peers cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635), 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321)
I20260809 18:52:12.791579 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00a0be5af8b47858a7a80d4aaa2fa20" candidate_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6df2e2f66849e2a92907caba674e49"
I20260809 18:52:12.791595 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f00a0be5af8b47858a7a80d4aaa2fa20" candidate_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52332febb242438dbf08e727c4383e99"
I20260809 18:52:12.791667 4401 raft_consensus.cc:3037] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.791671 4831 raft_consensus.cc:3037] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:12.792363 4831 raft_consensus.cc:2454] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94d5c08ca7254b87b7ee2d2d5c63e271 in term 1.
I20260809 18:52:12.792363 4401 raft_consensus.cc:2454] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94d5c08ca7254b87b7ee2d2d5c63e271 in term 1.
I20260809 18:52:12.792542 4509 leader_election.cc:302] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94d5c08ca7254b87b7ee2d2d5c63e271, cb6df2e2f66849e2a92907caba674e49; no voters:
I20260809 18:52:12.792645 4973 raft_consensus.cc:2781] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:12.792711 4973 raft_consensus.cc:686] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 LEADER]: Becoming Leader. State: Replica: 94d5c08ca7254b87b7ee2d2d5c63e271, State: Running, Role: LEADER
I20260809 18:52:12.792768 4973 consensus_queue.cc:237] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.793345 4156 catalog_manager.cc:5495] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194). New cstate: current_term: 1 leader_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } health_report { overall_health: UNKNOWN } } }
I20260809 18:52:12.816895 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:12.814910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41480 (local address 127.23.218.194:39399)
0809 18:52:12.814957 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:12.814958 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:12.814968 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:52:12.814990 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:12.814991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:12.814999 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:12.815041 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:12.815137 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.815611 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.816328 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:12.816511 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:12.816562 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:12.816688 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:12.816713 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:12.816750 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:12.816827 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 18:52:12.816845 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:52:12.817576 4831 raft_consensus.cc:1261] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Refusing update from remote peer 94d5c08ca7254b87b7ee2d2d5c63e271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:12.817577 4401 raft_consensus.cc:1261] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Refusing update from remote peer 94d5c08ca7254b87b7ee2d2d5c63e271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:12.817821 4973 consensus_queue.cc:1035] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:12.817942 4970 consensus_queue.cc:1035] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:12.820142 4156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:12.820217 4156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:12.820763 4156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd: Sending DeleteTablet for 3 replicas of tablet f00a0be5af8b47858a7a80d4aaa2fa20
I20260809 18:52:12.820971 4363 tablet_service.cc:1514] Processing DeleteTablet for tablet f00a0be5af8b47858a7a80d4aaa2fa20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:12 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41404
I20260809 18:52:12.820990 4804 tablet_service.cc:1514] Processing DeleteTablet for tablet f00a0be5af8b47858a7a80d4aaa2fa20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:12 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:35484
I20260809 18:52:12.821058 4997 tablet_replica.cc:330] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: stopping tablet replica
I20260809 18:52:12.821098 4998 tablet_replica.cc:330] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: stopping tablet replica
I20260809 18:52:12.821056 4593 tablet_service.cc:1514] Processing DeleteTablet for tablet f00a0be5af8b47858a7a80d4aaa2fa20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:12 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41454
I20260809 18:52:12.821135 4997 raft_consensus.cc:2229] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:12.821147 4998 raft_consensus.cc:2229] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:12.821154 4999 tablet_replica.cc:330] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: stopping tablet replica
I20260809 18:52:12.821174 4997 raft_consensus.cc:2258] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:12.821187 4998 raft_consensus.cc:2258] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:12.821192 4999 raft_consensus.cc:2229] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:52:12.821240 4999 raft_consensus.cc:2258] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:12.821393 4156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 18:52:12.821481 4998 ts_tablet_manager.cc:1905] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 18:52:12.821521 4156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:12.821718 4997 ts_tablet_manager.cc:1905] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:12.821817 4999 ts_tablet_manager.cc:1905] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:12.822937 4998 ts_tablet_manager.cc:1918] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:12.822999 4998 log.cc:1196] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/f00a0be5af8b47858a7a80d4aaa2fa20
I20260809 18:52:12.823038 4999 ts_tablet_manager.cc:1918] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:12.823092 4999 log.cc:1196] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/f00a0be5af8b47858a7a80d4aaa2fa20
I20260809 18:52:12.823208 4998 ts_tablet_manager.cc:1939] T f00a0be5af8b47858a7a80d4aaa2fa20 P 52332febb242438dbf08e727c4383e99: Deleting consensus metadata
I20260809 18:52:12.823244 4997 ts_tablet_manager.cc:1918] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:12.823285 4997 log.cc:1196] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/f00a0be5af8b47858a7a80d4aaa2fa20
I20260809 18:52:12.823319 4999 ts_tablet_manager.cc:1939] T f00a0be5af8b47858a7a80d4aaa2fa20 P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting consensus metadata
I20260809 18:52:12.823470 4997 ts_tablet_manager.cc:1939] T f00a0be5af8b47858a7a80d4aaa2fa20 P cb6df2e2f66849e2a92907caba674e49: Deleting consensus metadata
I20260809 18:52:12.823534 4140 catalog_manager.cc:4841] TS 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321): tablet f00a0be5af8b47858a7a80d4aaa2fa20 (table test-table [id=6af557ad0a3e402c934a581eddc5dfd1]) successfully deleted
I20260809 18:52:12.823626 4142 catalog_manager.cc:4841] TS 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399): tablet f00a0be5af8b47858a7a80d4aaa2fa20 (table test-table [id=6af557ad0a3e402c934a581eddc5dfd1]) successfully deleted
I20260809 18:52:12.823734 4140 catalog_manager.cc:4841] TS cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635): tablet f00a0be5af8b47858a7a80d4aaa2fa20 (table test-table [id=6af557ad0a3e402c934a581eddc5dfd1]) successfully deleted
I20260809 18:52:12.824339 4593 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d (DEFAULT_TABLE table=test-table [id=8d3147124987439ab55eb9f79c06d2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.824353 4363 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d (DEFAULT_TABLE table=test-table [id=8d3147124987439ab55eb9f79c06d2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.824350 4804 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d (DEFAULT_TABLE table=test-table [id=8d3147124987439ab55eb9f79c06d2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:12.824448 4804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.824448 4363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.824448 4593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:12.825248 4964 tablet_bootstrap.cc:492] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: Bootstrap starting.
I20260809 18:52:12.825510 4961 tablet_bootstrap.cc:492] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: Bootstrap starting.
I20260809 18:52:12.825528 4964 tablet_bootstrap.cc:654] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.825517 4959 tablet_bootstrap.cc:492] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: Bootstrap starting.
I20260809 18:52:12.825785 4959 tablet_bootstrap.cc:654] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.825798 4961 tablet_bootstrap.cc:654] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:12.826087 4964 tablet_bootstrap.cc:492] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: No bootstrap required, opened a new log
I20260809 18:52:12.826135 4964 ts_tablet_manager.cc:1397] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 18:52:12.826233 4959 tablet_bootstrap.cc:492] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: No bootstrap required, opened a new log
I20260809 18:52:12.826226 4964 raft_consensus.cc:348] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.826267 4964 raft_consensus.cc:374] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.826265 4959 ts_tablet_manager.cc:1397] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 18:52:12.826280 4964 raft_consensus.cc:729] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d5c08ca7254b87b7ee2d2d5c63e271, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.826321 4964 consensus_queue.cc:260] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.826333 4959 raft_consensus.cc:348] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.826411 4959 raft_consensus.cc:374] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.826431 4959 raft_consensus.cc:729] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6df2e2f66849e2a92907caba674e49, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.826426 4964 ts_tablet_manager.cc:1428] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:12.826473 4959 consensus_queue.cc:260] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.826558 4959 ts_tablet_manager.cc:1428] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:12.826964 4961 tablet_bootstrap.cc:492] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: No bootstrap required, opened a new log
I20260809 18:52:12.827014 4961 ts_tablet_manager.cc:1397] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260809 18:52:12.827121 4961 raft_consensus.cc:348] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.827188 4961 raft_consensus.cc:374] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:12.827207 4961 raft_consensus.cc:729] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52332febb242438dbf08e727c4383e99, State: Initialized, Role: FOLLOWER
I20260809 18:52:12.827245 4961 consensus_queue.cc:260] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:12.827334 4961 ts_tablet_manager.cc:1428] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:13.079784 5046 raft_consensus.cc:482] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:13.079890 5046 raft_consensus.cc:504] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:13.080067 5046 leader_election.cc:288] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635), 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399)
I20260809 18:52:13.080300 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad0f2741e2340a3b2d1e0282a17ac2d" candidate_uuid: "52332febb242438dbf08e727c4383e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6df2e2f66849e2a92907caba674e49" is_pre_election: true
I20260809 18:52:13.080319 4624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad0f2741e2340a3b2d1e0282a17ac2d" candidate_uuid: "52332febb242438dbf08e727c4383e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" is_pre_election: true
I20260809 18:52:13.080394 4401 raft_consensus.cc:2454] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52332febb242438dbf08e727c4383e99 in term 0.
I20260809 18:52:13.080418 4624 raft_consensus.cc:2454] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52332febb242438dbf08e727c4383e99 in term 0.
I20260809 18:52:13.080592 4734 leader_election.cc:302] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52332febb242438dbf08e727c4383e99, 94d5c08ca7254b87b7ee2d2d5c63e271; no voters:
I20260809 18:52:13.080720 5046 raft_consensus.cc:2781] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:13.080780 5046 raft_consensus.cc:482] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:13.080808 5046 raft_consensus.cc:3037] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:13.081539 5046 raft_consensus.cc:504] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:13.081665 5046 leader_election.cc:288] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [CANDIDATE]: Term 1 election: Requested vote from peers cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635), 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399)
I20260809 18:52:13.081813 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad0f2741e2340a3b2d1e0282a17ac2d" candidate_uuid: "52332febb242438dbf08e727c4383e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6df2e2f66849e2a92907caba674e49"
I20260809 18:52:13.081818 4624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad0f2741e2340a3b2d1e0282a17ac2d" candidate_uuid: "52332febb242438dbf08e727c4383e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271"
I20260809 18:52:13.081885 4401 raft_consensus.cc:3037] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:13.081887 4624 raft_consensus.cc:3037] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:13.082566 4401 raft_consensus.cc:2454] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52332febb242438dbf08e727c4383e99 in term 1.
I20260809 18:52:13.082604 4624 raft_consensus.cc:2454] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52332febb242438dbf08e727c4383e99 in term 1.
I20260809 18:52:13.082739 4730 leader_election.cc:302] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52332febb242438dbf08e727c4383e99, cb6df2e2f66849e2a92907caba674e49; no voters:
I20260809 18:52:13.082849 5046 raft_consensus.cc:2781] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:13.082908 5046 raft_consensus.cc:686] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 1 LEADER]: Becoming Leader. State: Replica: 52332febb242438dbf08e727c4383e99, State: Running, Role: LEADER
I20260809 18:52:13.082954 5046 consensus_queue.cc:237] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } }
I20260809 18:52:13.083542 4156 catalog_manager.cc:5495] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52332febb242438dbf08e727c4383e99 (127.23.218.195). New cstate: current_term: 1 leader_uuid: "52332febb242438dbf08e727c4383e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52332febb242438dbf08e727c4383e99" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41321 } health_report { overall_health: HEALTHY } } }
I20260809 18:52:13.110143 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:13.108332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35514 (local address 127.23.218.195:41321)
0809 18:52:13.108371 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:13.108373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:13.108395 (+ 22us) server_negotiation.cc:407] Connection header received
0809 18:52:13.108413 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:13.108414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:13.108422 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:13.108456 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:13.108542 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:13.109008 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:13.109659 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:13.109825 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:13.109867 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:13.109961 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:13.109985 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:13.110012 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:13.110048 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 18:52:13.110063 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:52:13.114032 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:13.112139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:49645 (local address 127.23.218.195:41321)
0809 18:52:13.112171 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:13.112173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:13.112180 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:13.112232 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:13.112233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:13.112240 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:13.112287 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:13.112376 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:13.112794 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:13.113442 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:13.113613 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:13.113663 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:13.113789 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:13.113814 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:13.113852 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:13.113907 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 18:52:13.113958 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":16}
I20260809 18:52:13.114078 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:13.112055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41321 (local address 127.23.218.195:49645)
0809 18:52:13.112131 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:13.112141 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:13.112190 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:13.112291 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:13.112292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:13.112293 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 18:52:13.112326 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:13.112327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:13.112864 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:13.112865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:13.113364 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:13.113365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:13.113676 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:13.113677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:13.113726 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:13.113730 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 18:52:13.113865 (+ 135us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 18:52:13.113867 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:52:13.113906 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 18:52:13.113983 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":31}
I20260809 18:52:13.115821 4401 raft_consensus.cc:1261] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Refusing update from remote peer 52332febb242438dbf08e727c4383e99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:13.115829 4624 raft_consensus.cc:1261] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Refusing update from remote peer 52332febb242438dbf08e727c4383e99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:13.116061 5046 consensus_queue.cc:1035] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94d5c08ca7254b87b7ee2d2d5c63e271" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 39399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:13.116159 4968 consensus_queue.cc:1035] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6df2e2f66849e2a92907caba674e49" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:13.126331 4156 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:13.126423 4156 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:13.126979 4156 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97053079adc7491fa529722ee712cdbd: Sending DeleteTablet for 3 replicas of tablet 1ad0f2741e2340a3b2d1e0282a17ac2d
I20260809 18:52:13.127221 4593 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:13 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41454
I20260809 18:52:13.127233 4363 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:13 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41404
I20260809 18:52:13.127311 4999 tablet_replica.cc:330] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: stopping tablet replica
I20260809 18:52:13.127315 4997 tablet_replica.cc:330] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: stopping tablet replica
I20260809 18:52:13.127308 4804 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:13 UTC) from {username='oryx', principal='oryx/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:35484
I20260809 18:52:13.127354 4997 raft_consensus.cc:2229] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:13.127359 4999 raft_consensus.cc:2229] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:13.127386 4997 raft_consensus.cc:2258] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:13.127385 4998 tablet_replica.cc:330] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: stopping tablet replica
I20260809 18:52:13.127390 4999 raft_consensus.cc:2258] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:13.127429 4998 raft_consensus.cc:2229] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:52:13.127507 4998 raft_consensus.cc:2258] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:13.127760 4998 ts_tablet_manager.cc:1905] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:13.127759 4999 ts_tablet_manager.cc:1905] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:13.127810 4997 ts_tablet_manager.cc:1905] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:13.129336 4999 ts_tablet_manager.cc:1918] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:52:13.129369 4998 ts_tablet_manager.cc:1918] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:52:13.129395 4999 log.cc:1196] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/1ad0f2741e2340a3b2d1e0282a17ac2d
I20260809 18:52:13.129418 4998 log.cc:1196] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/1ad0f2741e2340a3b2d1e0282a17ac2d
I20260809 18:52:13.129599 4999 ts_tablet_manager.cc:1939] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 94d5c08ca7254b87b7ee2d2d5c63e271: Deleting consensus metadata
I20260809 18:52:13.129599 4998 ts_tablet_manager.cc:1939] T 1ad0f2741e2340a3b2d1e0282a17ac2d P 52332febb242438dbf08e727c4383e99: Deleting consensus metadata
I20260809 18:52:13.129884 4997 ts_tablet_manager.cc:1918] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 18:52:13.129902 4142 catalog_manager.cc:4841] TS 94d5c08ca7254b87b7ee2d2d5c63e271 (127.23.218.194:39399): tablet 1ad0f2741e2340a3b2d1e0282a17ac2d (table test-table [id=8d3147124987439ab55eb9f79c06d2fb]) successfully deleted
I20260809 18:52:13.129938 4997 log.cc:1196] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/1ad0f2741e2340a3b2d1e0282a17ac2d
I20260809 18:52:13.129930 4140 catalog_manager.cc:4841] TS 52332febb242438dbf08e727c4383e99 (127.23.218.195:41321): tablet 1ad0f2741e2340a3b2d1e0282a17ac2d (table test-table [id=8d3147124987439ab55eb9f79c06d2fb]) successfully deleted
I20260809 18:52:13.130118 4997 ts_tablet_manager.cc:1939] T 1ad0f2741e2340a3b2d1e0282a17ac2d P cb6df2e2f66849e2a92907caba674e49: Deleting consensus metadata
I20260809 18:52:13.130147 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 4268
W20260809 18:52:13.142416 4140 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:35635 (error 108)
I20260809 18:52:13.142925 5095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:13.142749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:35635 (local address 127.0.0.1:41424)
0809 18:52:13.142839 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:13.142892 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:35635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":39,"threads_started":1}
I20260809 18:52:13.143069 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 4485
W20260809 18:52:13.143074 4140 catalog_manager.cc:4584] TS cb6df2e2f66849e2a92907caba674e49 (127.23.218.193:35635): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ad0f2741e2340a3b2d1e0282a17ac2d: Network error: Client connection negotiation failed: client connection to 127.23.218.193:35635: connect: Connection refused (error 111)
I20260809 18:52:13.154892 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 4706
I20260809 18:52:13.167347 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 4113
2026-08-09T18:52:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2869 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:52:13 dist-test-slave-6zbq krb5kdc[5098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:52:13 dist-test-slave-6zbq krb5kdc[5098](info): set up 2 sockets
Aug 09 18:52:13 dist-test-slave-6zbq krb5kdc[5098](info): commencing operation
krb5kdc: starting...
W20260809 18:52:14.316704 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:52:14 dist-test-slave-6zbq krb5kdc[5098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:52:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:52:14.432276 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:35563
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:41747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:35563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:14.519372 5117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:14.519551 5117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:14.519572 5117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:14.520613 5117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:14.520645 5117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:14.520658 5117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:14.520668 5117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:14.520678 5117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:14.522083 5117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:35563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:35563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:14.522272 5117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:14.522475 5117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:14.526993 5123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:14.527145 5117 server_base.cc:1034] running on GCE node
W20260809 18:52:14.527005 5122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:14.527068 5125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:14.527487 5117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:14.527745 5117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:14.528898 5117 hybrid_clock.cc:648] HybridClock initialized: now 1786301534528885 us; error 26 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T18:52:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1362 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:52:14 dist-test-slave-6zbq krb5kdc[5133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:52:14 dist-test-slave-6zbq krb5kdc[5133](info): set up 2 sockets
Aug 09 18:52:14 dist-test-slave-6zbq krb5kdc[5133](info): commencing operation
krb5kdc: starting...
W20260809 18:52:15.667309 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:52:15 dist-test-slave-6zbq krb5kdc[5133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:52:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:52:15.775041 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36389
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:36389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:15.853199 5157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:15.853384 5157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:15.853412 5157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:15.854677 5157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:15.854732 5157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:15.854755 5157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:15.854774 5157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:15.854789 5157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:15.856357 5157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:36389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:15.856588 5157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:15.856788 5157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:15.861292 5167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:15.861334 5157 server_base.cc:1034] running on GCE node
W20260809 18:52:15.861303 5173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:15.861374 5168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:15.861709 5157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:15.861967 5157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:15.863114 5157 hybrid_clock.cc:648] HybridClock initialized: now 1786301535863103 us; error 27 us; skew 500 ppm
Aug 09 18:52:15 dist-test-slave-6zbq krb5kdc[5133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301535, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:15.869379 5157 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:52:15.869773 5157 webserver.cc:460] Webserver started at http://127.23.218.254:38741/ using document root <none> and password file <none>
I20260809 18:52:15.869979 5157 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:15.870026 5157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:15.870150 5157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:15.871143 5157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "c39eef4161464759a6dbfc7dcfc050a4"
format_stamp: "Formatted at 2026-08-09 18:52:15 on dist-test-slave-6zbq"
server_key: "d880fb6bf0105f58c2662cf7aa503216"
server_key_iv: "62ada78827f8c051b57039561f3dd99a"
server_key_version: "encryptionkey@0"
I20260809 18:52:15.871459 5157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "c39eef4161464759a6dbfc7dcfc050a4"
format_stamp: "Formatted at 2026-08-09 18:52:15 on dist-test-slave-6zbq"
server_key: "d880fb6bf0105f58c2662cf7aa503216"
server_key_iv: "62ada78827f8c051b57039561f3dd99a"
server_key_version: "encryptionkey@0"
I20260809 18:52:15.872655 5157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:15.873387 5184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:15.873610 5157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:15.873701 5157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "c39eef4161464759a6dbfc7dcfc050a4"
format_stamp: "Formatted at 2026-08-09 18:52:15 on dist-test-slave-6zbq"
server_key: "d880fb6bf0105f58c2662cf7aa503216"
server_key_iv: "62ada78827f8c051b57039561f3dd99a"
server_key_version: "encryptionkey@0"
I20260809 18:52:15.873772 5157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:15.880869 5157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:15.882540 5157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:15.882665 5157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:15.886149 5157 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:36389
I20260809 18:52:15.886204 5282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:36389 every 8 connection(s)
I20260809 18:52:15.886515 5157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:15.887298 5284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:15.888643 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 5157
I20260809 18:52:15.888736 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:15.888899 24427 external_mini_cluster.cc:1428] Setting key f2aad141da3a7572e84c06dd807a183c
I20260809 18:52:15.889571 5284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: Bootstrap starting.
I20260809 18:52:15.890232 5284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:15.890560 5284 log.cc:824] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:15.891299 5284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: No bootstrap required, opened a new log
I20260809 18:52:15.893417 5284 raft_consensus.cc:348] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } }
I20260809 18:52:15.893573 5284 raft_consensus.cc:374] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:15.893605 5284 raft_consensus.cc:729] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c39eef4161464759a6dbfc7dcfc050a4, State: Initialized, Role: FOLLOWER
I20260809 18:52:15.893739 5284 consensus_queue.cc:260] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } }
I20260809 18:52:15.893821 5284 raft_consensus.cc:388] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:15.893851 5284 raft_consensus.cc:482] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:15.893898 5284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:15.894402 5284 raft_consensus.cc:504] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } }
I20260809 18:52:15.894510 5284 leader_election.cc:302] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c39eef4161464759a6dbfc7dcfc050a4; no voters:
I20260809 18:52:15.894709 5284 leader_election.cc:288] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:15.894765 5294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:15.894923 5294 raft_consensus.cc:686] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 1 LEADER]: Becoming Leader. State: Replica: c39eef4161464759a6dbfc7dcfc050a4, State: Running, Role: LEADER
I20260809 18:52:15.894958 5284 sys_catalog.cc:564] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:15.895018 5294 consensus_queue.cc:237] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } }
I20260809 18:52:15.895339 5294 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c39eef4161464759a6dbfc7dcfc050a4. Latest consensus state: current_term: 1 leader_uuid: "c39eef4161464759a6dbfc7dcfc050a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } } }
I20260809 18:52:15.895412 5294 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:15.895562 5295 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c39eef4161464759a6dbfc7dcfc050a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } } }
I20260809 18:52:15.895622 5295 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:15.895757 5301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:15.896219 5301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 18:52:15 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:15.899675 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:15.889378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47288 (local address 127.23.218.254:36389)
0809 18:52:15.889553 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:15.889562 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:15.889593 (+ 31us) server_negotiation.cc:407] Connection header received
0809 18:52:15.890117 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:15.890125 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:15.890196 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:15.890368 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:15.890996 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:15.892340 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:15.893087 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:15.893679 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:15.893788 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:15.896805 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:15.896809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:15.896809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:15.896819 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:15.898230 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:15.898648 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:15.898651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:15.898652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:15.898730 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:15.898948 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:15.898949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:15.898949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:15.899142 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:15.899241 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:15.899390 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 18:52:15.899484 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260809 18:52:15.903255 5301 catalog_manager.cc:1347] Generated new cluster ID: a43b57743ba04b499acdea8bb9f37bc5
I20260809 18:52:15.903312 5301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:15.922957 5301 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:15.923434 5301 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:15.926712 5301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: Generated new TSK 0
I20260809 18:52:15.926854 5301 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:52:15.959180 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36389
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:16.041851 5331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:16.042029 5331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:16.042049 5331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:16.043172 5331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:16.043212 5331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:16.043237 5331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:16.044641 5331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:16.044850 5331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:16.045056 5331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:16.049618 5342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.049623 5343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.049652 5347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:16.049769 5331 server_base.cc:1034] running on GCE node
I20260809 18:52:16.050004 5331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:16.050239 5331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:16.051399 5331 hybrid_clock.cc:648] HybridClock initialized: now 1786301536051392 us; error 37 us; skew 500 ppm
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:16.053715 5331 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:52:16.054106 5331 webserver.cc:460] Webserver started at http://127.23.218.193:39075/ using document root <none> and password file <none>
I20260809 18:52:16.054272 5331 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:16.054309 5331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:16.054435 5331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:16.055349 5331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "7b8e83c751e6430b9b770817371dba28"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "1446b50c2265af9c2044e93bc0702111"
server_key_iv: "83c4386cac21e90e410a97f18f88a8fc"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.055642 5331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "7b8e83c751e6430b9b770817371dba28"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "1446b50c2265af9c2044e93bc0702111"
server_key_iv: "83c4386cac21e90e410a97f18f88a8fc"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.056734 5331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:52:16.057348 5358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.057510 5331 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:16.057587 5331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "7b8e83c751e6430b9b770817371dba28"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "1446b50c2265af9c2044e93bc0702111"
server_key_iv: "83c4386cac21e90e410a97f18f88a8fc"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.057639 5331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:16.067297 5331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:16.069133 5331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:16.069233 5331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:16.069387 5331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:16.069608 5331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:16.069638 5331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.069664 5331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:16.069675 5331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.074863 5331 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:41415
I20260809 18:52:16.074931 5546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:41415 every 8 connection(s)
I20260809 18:52:16.075165 5331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:16.083076 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.075775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:59961 (local address 127.23.218.254:36389)
0809 18:52:16.075835 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:16.075837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:16.076016 (+ 179us) server_negotiation.cc:407] Connection header received
0809 18:52:16.076470 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:16.076471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:16.076479 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:16.076529 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:16.077342 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.077840 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.078759 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.078943 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.078989 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.080771 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:16.080773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:16.080773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:16.080779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:16.082002 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:16.082330 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:16.082331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:16.082331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:16.082406 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:16.082608 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:16.082609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:16.082609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:16.082707 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:16.082751 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:16.082948 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 18:52:16.083015 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 18:52:16.083202 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.075740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:59961)
0809 18:52:16.075904 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:16.075941 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:16.076392 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:16.076576 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:16.076579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:16.077074 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:16.077258 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.077265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.077909 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.077910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.078697 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.078700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.079003 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.079004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.079093 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.079457 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:16.079465 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:16.080692 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:16.082070 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:16.082074 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:16.082076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:16.082271 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:16.082458 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:16.082458 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:16.082459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:16.082556 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:16.082753 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:16.082756 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:16.082854 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 18:52:16.082927 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 18:52:16.083035 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260809 18:52:16.083678 5548 heartbeater.cc:347] Connected to a master server at 127.23.218.254:36389
I20260809 18:52:16.083755 5548 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:16.083880 5548 heartbeater.cc:511] Master 127.23.218.254:36389 requested a full tablet report, sending...
I20260809 18:52:16.084324 5217 ts_manager.cc:194] Registered new tserver with Master: 7b8e83c751e6430b9b770817371dba28 (127.23.218.193:41415)
I20260809 18:52:16.084352 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 5331
I20260809 18:52:16.084430 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:52:16.084606 24427 external_mini_cluster.cc:1428] Setting key 3e6c9f26084f85b60a6ec311ea5a0b3b
I20260809 18:52:16.085129 5217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:59961
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:52:16.108466 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36389
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:16.186244 5554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:16.186470 5554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:16.186503 5554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:16.187604 5554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:16.187641 5554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:16.187675 5554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:16.189073 5554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:16.189301 5554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:16.189499 5554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:16.193966 5563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.194005 5562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.194072 5565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:16.194079 5554 server_base.cc:1034] running on GCE node
I20260809 18:52:16.194329 5554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:16.194620 5554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:16.195747 5554 hybrid_clock.cc:648] HybridClock initialized: now 1786301536195740 us; error 26 us; skew 500 ppm
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:16.198045 5554 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:52:16.198498 5554 webserver.cc:460] Webserver started at http://127.23.218.194:37625/ using document root <none> and password file <none>
I20260809 18:52:16.198704 5554 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:16.198745 5554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:16.198863 5554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:16.199798 5554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "6e531e70410b437a82cbc4818f15053c"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "645800c791ad19efc247529e1642aa62"
server_key_iv: "615b0be2be899e90f4997877fb77d1aa"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.200109 5554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "6e531e70410b437a82cbc4818f15053c"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "645800c791ad19efc247529e1642aa62"
server_key_iv: "615b0be2be899e90f4997877fb77d1aa"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.201280 5554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:52:16.201882 5574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.202080 5554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:16.202160 5554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "6e531e70410b437a82cbc4818f15053c"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "645800c791ad19efc247529e1642aa62"
server_key_iv: "615b0be2be899e90f4997877fb77d1aa"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.202210 5554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:16.218855 5554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:16.220585 5554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:16.220688 5554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:16.220846 5554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:16.221094 5554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:16.221125 5554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.221143 5554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:16.221166 5554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.226235 5554 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:41611
I20260809 18:52:16.226269 5745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:41611 every 8 connection(s)
I20260809 18:52:16.226545 5554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:16.232646 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 5554
I20260809 18:52:16.232749 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:52:16.232939 24427 external_mini_cluster.cc:1428] Setting key 4e722aedbb8733c5e86d78b43c688048
I20260809 18:52:16.234798 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.227163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:52337 (local address 127.23.218.254:36389)
0809 18:52:16.227239 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:16.227241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:16.227392 (+ 151us) server_negotiation.cc:407] Connection header received
0809 18:52:16.227800 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:16.227801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:16.227811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:16.227865 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:16.228701 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.229210 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.230080 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.230257 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.230300 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.232173 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:16.232174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:16.232174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:16.232181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:16.233499 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:16.233917 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:16.233918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:16.233919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:16.233974 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:16.234234 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:16.234235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:16.234235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:16.234332 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:16.234411 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:16.234672 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 18:52:16.234737 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 18:52:16.234982 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.227123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:52337)
0809 18:52:16.227286 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:16.227318 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:16.227724 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:16.227912 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:16.227915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:16.228430 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:16.228597 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.228602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.229283 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.229283 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.230017 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.230019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.230314 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.230315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.230464 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.230881 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:16.230889 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:16.232092 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:16.233565 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:16.233572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:16.233574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:16.233840 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:16.234039 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:16.234040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:16.234040 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:16.234175 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:16.234400 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:16.234405 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:16.234545 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 18:52:16.234647 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 18:52:16.234792 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260809 18:52:16.235446 5747 heartbeater.cc:347] Connected to a master server at 127.23.218.254:36389
I20260809 18:52:16.235526 5747 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:16.235682 5747 heartbeater.cc:511] Master 127.23.218.254:36389 requested a full tablet report, sending...
I20260809 18:52:16.236025 5217 ts_manager.cc:194] Registered new tserver with Master: 6e531e70410b437a82cbc4818f15053c (127.23.218.194:41611)
I20260809 18:52:16.236488 5217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:52337
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:52:16.256773 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36389
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:16.335824 5753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:16.336009 5753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:16.336031 5753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:16.337109 5753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:16.337147 5753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:16.337172 5753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:16.338521 5753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:16.338701 5753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:16.338861 5753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:16.343470 5763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.343631 5760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.343652 5766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:16.343719 5753 server_base.cc:1034] running on GCE node
I20260809 18:52:16.343926 5753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:16.344163 5753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:16.345340 5753 hybrid_clock.cc:648] HybridClock initialized: now 1786301536345325 us; error 29 us; skew 500 ppm
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:16.347662 5753 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:52:16.348088 5753 webserver.cc:460] Webserver started at http://127.23.218.195:42253/ using document root <none> and password file <none>
I20260809 18:52:16.348284 5753 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:16.348330 5753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:16.348446 5753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:16.349385 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "4056fb6bb6964e4196989bff8e8aba54"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "a3eb0b2e0777152a136f25933254d621"
server_key_iv: "5bbc6abf1bd5319977fdc83dc898cebf"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.349694 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "4056fb6bb6964e4196989bff8e8aba54"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "a3eb0b2e0777152a136f25933254d621"
server_key_iv: "5bbc6abf1bd5319977fdc83dc898cebf"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.350992 5753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:52:16.351629 5777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.351790 5753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:16.351872 5753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "4056fb6bb6964e4196989bff8e8aba54"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "a3eb0b2e0777152a136f25933254d621"
server_key_iv: "5bbc6abf1bd5319977fdc83dc898cebf"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.351934 5753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:16.364252 5753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:16.365777 5753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:16.365899 5753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:16.366086 5753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:16.366407 5753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:16.366451 5753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.366482 5753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:16.366505 5753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.371862 5753 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:45557
I20260809 18:52:16.371924 5946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:45557 every 8 connection(s)
I20260809 18:52:16.372170 5753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:16.380338 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.372826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:36913 (local address 127.23.218.254:36389)
0809 18:52:16.372897 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:16.372899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:16.373054 (+ 155us) server_negotiation.cc:407] Connection header received
0809 18:52:16.373548 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:16.373549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:16.373560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:16.373610 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:16.374491 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.374987 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.375947 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.376148 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.376203 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.378034 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:16.378035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:16.378036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:16.378043 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:16.379136 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:16.379493 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:16.379494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:16.379494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:16.379545 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:16.379767 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:16.379768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:16.379768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:16.379871 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:16.379927 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:16.380179 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 18:52:16.380253 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 18:52:16.380483 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.372777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:36913)
0809 18:52:16.372946 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:16.372977 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:16.373451 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:16.373673 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:16.373676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:16.374186 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:16.374409 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.374414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.375075 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.375076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.375829 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.375831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.376225 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.376226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.376337 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.376695 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:16.376702 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:16.377941 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:16.379214 (+ 1273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:16.379220 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:16.379222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:16.379421 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:16.379605 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:16.379606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:16.379606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:16.379699 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:16.379936 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:16.379943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:16.380058 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 18:52:16.380141 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 18:52:16.380290 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260809 18:52:16.380934 5948 heartbeater.cc:347] Connected to a master server at 127.23.218.254:36389
I20260809 18:52:16.381027 5948 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:16.381122 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 5753
I20260809 18:52:16.381218 5948 heartbeater.cc:511] Master 127.23.218.254:36389 requested a full tablet report, sending...
I20260809 18:52:16.381227 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:52:16.381417 24427 external_mini_cluster.cc:1428] Setting key 89c121042d5d3f0039450fb9187efc0b
I20260809 18:52:16.381621 5217 ts_manager.cc:194] Registered new tserver with Master: 4056fb6bb6964e4196989bff8e8aba54 (127.23.218.195:45557)
I20260809 18:52:16.382063 5217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:36913
I20260809 18:52:16.382333 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab.
Entry for principal oryx/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab.
WARNING: no policy specified for oryx/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab.
Entry for principal oryx/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab.
WARNING: no policy specified for oryx/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab.
Entry for principal oryx/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab.
I20260809 18:52:16.444895 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 5331
I20260809 18:52:16.468914 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 5554
I20260809 18:52:16.479135 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 5753
I20260809 18:52:16.488082 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 5157
I20260809 18:52:16.494506 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36389
--webserver_interface=127.23.218.254
--webserver_port=38741
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:36389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:16.572431 5964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:16.572610 5964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:16.572640 5964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:16.573741 5964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:16.573782 5964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:16.573801 5964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:16.573817 5964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:16.573832 5964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:16.575268 5964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:36389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=38741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:16.575493 5964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:16.575671 5964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:16.580070 5976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.580113 5970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.580070 5974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:16.580219 5964 server_base.cc:1034] running on GCE node
I20260809 18:52:16.580497 5964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:16.580762 5964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:16.581907 5964 hybrid_clock.cc:648] HybridClock initialized: now 1786301536581886 us; error 34 us; skew 500 ppm
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:16.584136 5964 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:52:16.584553 5964 webserver.cc:460] Webserver started at http://127.23.218.254:38741/ using document root <none> and password file <none>
I20260809 18:52:16.584751 5964 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:16.584812 5964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:16.586037 5964 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:16.586664 5987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.586817 5964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.586899 5964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "c39eef4161464759a6dbfc7dcfc050a4"
format_stamp: "Formatted at 2026-08-09 18:52:15 on dist-test-slave-6zbq"
server_key: "d880fb6bf0105f58c2662cf7aa503216"
server_key_iv: "62ada78827f8c051b57039561f3dd99a"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.587184 5964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:16.607441 5964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:16.609779 5964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:16.609918 5964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:16.614154 5964 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:36389
I20260809 18:52:16.614195 6065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:36389 every 8 connection(s)
I20260809 18:52:16.614566 5964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:16.615873 6066 sys_catalog.cc:263] Verifying existing consensus state
I20260809 18:52:16.616484 6066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: Bootstrap starting.
I20260809 18:52:16.618726 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 5964
I20260809 18:52:16.618981 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:41415
--local_ip_for_outbound_sockets=127.23.218.193
--tserver_master_addrs=127.23.218.254:36389
--webserver_port=39075
--webserver_interface=127.23.218.193
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.218.193
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:52:16.624445 6066 log.cc:824] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:16.625962 6066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:52:16.626215 6066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: Bootstrap complete.
I20260809 18:52:16.628221 6066 raft_consensus.cc:348] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } }
I20260809 18:52:16.628432 6066 raft_consensus.cc:729] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c39eef4161464759a6dbfc7dcfc050a4, State: Initialized, Role: FOLLOWER
I20260809 18:52:16.628566 6066 consensus_queue.cc:260] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } }
I20260809 18:52:16.628641 6066 raft_consensus.cc:388] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:16.628685 6066 raft_consensus.cc:482] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:16.628729 6066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 18:52:16.629644 6066 raft_consensus.cc:504] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } }
I20260809 18:52:16.629730 6066 leader_election.cc:302] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c39eef4161464759a6dbfc7dcfc050a4; no voters:
I20260809 18:52:16.629881 6066 leader_election.cc:288] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 18:52:16.629936 6072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 18:52:16.630059 6072 raft_consensus.cc:686] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [term 2 LEADER]: Becoming Leader. State: Replica: c39eef4161464759a6dbfc7dcfc050a4, State: Running, Role: LEADER
I20260809 18:52:16.630154 6066 sys_catalog.cc:564] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:16.630155 6072 consensus_queue.cc:237] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } }
I20260809 18:52:16.630497 6075 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c39eef4161464759a6dbfc7dcfc050a4. Latest consensus state: current_term: 2 leader_uuid: "c39eef4161464759a6dbfc7dcfc050a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } } }
I20260809 18:52:16.630510 6073 sys_catalog.cc:455] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c39eef4161464759a6dbfc7dcfc050a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c39eef4161464759a6dbfc7dcfc050a4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36389 } } }
I20260809 18:52:16.630625 6075 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:16.630635 6073 sys_catalog.cc:458] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:16.631067 6079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:16.631707 6079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:16.631830 6079 catalog_manager.cc:1259] Loaded cluster ID: a43b57743ba04b499acdea8bb9f37bc5
I20260809 18:52:16.631855 6079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:16.633399 6079 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:16.635094 6079 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c39eef4161464759a6dbfc7dcfc050a4: Loaded TSK: 0
I20260809 18:52:16.636818 6079 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 18:52:16.703436 6070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:16.703624 6070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:16.703645 6070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:16.704775 6070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:16.704813 6070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:16.704838 6070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:16.706235 6070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.193.keytab
--principal=oryx/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:41415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=39075
--tserver_master_addrs=127.23.218.254:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:16.706452 6070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:16.706674 6070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:16.711419 6111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.711405 6108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.711570 6109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:16.711855 6070 server_base.cc:1034] running on GCE node
I20260809 18:52:16.712049 6070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:16.712285 6070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:16.713450 6070 hybrid_clock.cc:648] HybridClock initialized: now 1786301536713439 us; error 26 us; skew 500 ppm
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:16.715996 6070 init.cc:377] Logged in from keytab as oryx/127.23.218.193@KRBTEST.COM (short username oryx)
I20260809 18:52:16.716449 6070 webserver.cc:460] Webserver started at http://127.23.218.193:39075/ using document root <none> and password file <none>
I20260809 18:52:16.716671 6070 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:16.716741 6070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:16.718031 6070 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:16.718652 6121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.718849 6070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:16.718952 6070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "7b8e83c751e6430b9b770817371dba28"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "1446b50c2265af9c2044e93bc0702111"
server_key_iv: "83c4386cac21e90e410a97f18f88a8fc"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.719403 6070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:16.752027 6070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:16.754302 6070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:16.754469 6070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:16.754704 6070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:16.755016 6070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:16.755056 6070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.755093 6070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:16.755117 6070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.761837 6070 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:41415
I20260809 18:52:16.761902 6292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:41415 every 8 connection(s)
I20260809 18:52:16.762228 6070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:52:16.764746 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 6070
I20260809 18:52:16.765053 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:41611
--local_ip_for_outbound_sockets=127.23.218.194
--tserver_master_addrs=127.23.218.254:36389
--webserver_port=37625
--webserver_interface=127.23.218.194
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.218.194
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:16.770571 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.762905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:34001)
0809 18:52:16.763067 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:16.763105 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:16.763602 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:16.764050 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:16.764054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:16.764654 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:16.764920 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.764928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.766066 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.766067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.767009 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.767012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.767746 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.767747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.767844 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.768271 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:16.768279 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:16.770441 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
W20260809 18:52:16.770766 6299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:16.762997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:34001 (local address 127.23.218.254:36389)
0809 18:52:16.763183 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:16.763191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:16.763229 (+ 38us) server_negotiation.cc:407] Connection header received
0809 18:52:16.763819 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:16.763825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:16.763877 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:16.763991 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:16.765049 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.765956 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.767092 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.767668 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.767775 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.770598 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:34001: BlockingRecv error: recv got EOF from 127.23.218.193:34001 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
W20260809 18:52:16.770769 6296 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:36389 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.218.254:36389: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:16.774879 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.770962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:58287)
0809 18:52:16.771093 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:16.771101 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:16.771143 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:16.771251 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:16.771252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:16.771364 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:16.771407 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.771409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.771999 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.772000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.772484 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.772485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.772817 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.772818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.772877 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.773210 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:16.773215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:16.774823 (+ 1608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
W20260809 18:52:16.775170 6299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:16.771021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:58287 (local address 127.23.218.254:36389)
0809 18:52:16.771077 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:16.771079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:16.771168 (+ 89us) server_negotiation.cc:407] Connection header received
0809 18:52:16.771194 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:16.771195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:16.771206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:16.771251 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:16.771479 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.771925 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.772564 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.772751 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.772804 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.774915 (+ 2111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:58287: BlockingRecv error: recv got EOF from 127.23.218.193:58287 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:16.778728 6298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.775088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:40203)
0809 18:52:16.775131 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:16.775140 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:16.775175 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:16.775333 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:16.775334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:16.775438 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:16.775461 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.775462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.776028 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.776029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.776479 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.776480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.776801 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.776802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.776847 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.777157 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:16.777161 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:16.778678 (+ 1517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 18:52:16.778805 6299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:16.775159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:40203 (local address 127.23.218.254:36389)
0809 18:52:16.775247 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:16.775249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:16.775255 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:52:16.775274 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:16.775275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:16.775284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:16.775331 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:16.775532 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.775955 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.776553 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.776737 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.776788 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.778756 (+ 1968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:40203: BlockingRecv error: recv got EOF from 127.23.218.193:40203 (error 108)
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":79,"threads_started":1}
W20260809 18:52:16.779013 6296 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 18:52:16.844630 6300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:16.844810 6300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:16.844831 6300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:16.845920 6300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:16.845958 6300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:16.845986 6300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:16.847376 6300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.194.keytab
--principal=oryx/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:41611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=37625
--tserver_master_addrs=127.23.218.254:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:16.847577 6300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:16.847777 6300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:16.852290 6310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.852286 6311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.852284 6314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:16.852561 6300 server_base.cc:1034] running on GCE node
I20260809 18:52:16.852753 6300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:16.852969 6300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:16.854105 6300 hybrid_clock.cc:648] HybridClock initialized: now 1786301536854089 us; error 33 us; skew 500 ppm
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:16.856318 6300 init.cc:377] Logged in from keytab as oryx/127.23.218.194@KRBTEST.COM (short username oryx)
I20260809 18:52:16.856726 6300 webserver.cc:460] Webserver started at http://127.23.218.194:37625/ using document root <none> and password file <none>
I20260809 18:52:16.856928 6300 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:16.856987 6300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:16.858201 6300 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:16.858762 6324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.858920 6300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.858999 6300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "6e531e70410b437a82cbc4818f15053c"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "645800c791ad19efc247529e1642aa62"
server_key_iv: "615b0be2be899e90f4997877fb77d1aa"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.859299 6300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:16.869202 6300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:16.870812 6300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:16.870934 6300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:16.871114 6300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:16.871380 6300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:16.871418 6300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.871443 6300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:16.871459 6300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.876761 6300 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:41611
I20260809 18:52:16.876822 6507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:41611 every 8 connection(s)
I20260809 18:52:16.877090 6300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:52:16.880105 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 6300
I20260809 18:52:16.880353 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:45557
--local_ip_for_outbound_sockets=127.23.218.195
--tserver_master_addrs=127.23.218.254:36389
--webserver_port=42253
--webserver_interface=127.23.218.195
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.218.195
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 18:52:16.883821 6299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:16.877664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:41013 (local address 127.23.218.254:36389)
0809 18:52:16.877705 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:16.877707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:16.877918 (+ 211us) server_negotiation.cc:407] Connection header received
0809 18:52:16.878313 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:16.878314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:16.878326 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:16.878404 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:16.879232 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.879709 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.880719 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.880984 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.881056 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.883768 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:41013: BlockingRecv error: recv got EOF from 127.23.218.194:41013 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:52:16.883839 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.877644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:41013)
0809 18:52:16.877798 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:16.877836 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:16.878240 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:16.878472 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:16.878475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:16.878976 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:16.879143 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.879147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.879779 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.879780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.880633 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.880636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.881059 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.881059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.881165 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.881567 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:16.881574 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:16.883665 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
W20260809 18:52:16.884059 6513 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:36389 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.218.254:36389: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:16.888468 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.884317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:53827)
0809 18:52:16.884406 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:16.884415 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:16.884460 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:16.884576 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:16.884577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:16.884683 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:16.884723 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.884724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.885536 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.885537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.886003 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.886004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.886452 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.886452 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.886503 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.886881 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:16.886885 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:16.888410 (+ 1525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 18:52:16.888621 6299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:16.884365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:53827 (local address 127.23.218.254:36389)
0809 18:52:16.884427 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:16.884429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:16.884452 (+ 23us) server_negotiation.cc:407] Connection header received
0809 18:52:16.884499 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:16.884500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:16.884510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:16.884568 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:16.884786 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.885459 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.886081 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.886383 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.886451 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.888564 (+ 2113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:53827: BlockingRecv error: recv got EOF from 127.23.218.194:53827 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:16.892829 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:16.888676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:41699)
0809 18:52:16.888764 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:16.888773 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:16.888817 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:16.888953 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:16.888954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:16.889080 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:16.889111 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.889113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.889719 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.889720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.890475 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:16.890478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.890791 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.890792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:16.890842 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.891123 (+ 281us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:16.891128 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:16.892776 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 18:52:16.892937 6299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:16.888752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:41699 (local address 127.23.218.254:36389)
0809 18:52:16.888780 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:16.888782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:16.888806 (+ 24us) server_negotiation.cc:407] Connection header received
0809 18:52:16.888871 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:16.888872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:16.888879 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:16.888922 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:16.889209 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.889636 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.890545 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:16.890718 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:16.890765 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:16.892886 (+ 2121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:41699: BlockingRecv error: recv got EOF from 127.23.218.194:41699 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260809 18:52:16.892948 6513 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 18:52:16.960736 6516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:16.960912 6516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:16.960942 6516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:16.962117 6516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:16.962157 6516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:16.962191 6516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:16.963630 6516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/krb5kdc/oryx_127.23.218.195.keytab
--principal=oryx/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:45557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=42253
--tserver_master_addrs=127.23.218.254:36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:16.963850 6516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:16.964051 6516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:16.968689 6522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.968698 6524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:16.968699 6526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:16.968997 6516 server_base.cc:1034] running on GCE node
I20260809 18:52:16.969177 6516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:16.969362 6516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:16.970479 6516 hybrid_clock.cc:648] HybridClock initialized: now 1786301536970465 us; error 28 us; skew 500 ppm
Aug 09 18:52:16 dist-test-slave-6zbq krb5kdc[5133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301536, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:16.972757 6516 init.cc:377] Logged in from keytab as oryx/127.23.218.195@KRBTEST.COM (short username oryx)
I20260809 18:52:16.973130 6516 webserver.cc:460] Webserver started at http://127.23.218.195:42253/ using document root <none> and password file <none>
I20260809 18:52:16.973344 6516 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:16.973403 6516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:16.974614 6516 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:52:16.975090 6537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:16.975261 6516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:16.975345 6516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "4056fb6bb6964e4196989bff8e8aba54"
format_stamp: "Formatted at 2026-08-09 18:52:16 on dist-test-slave-6zbq"
server_key: "a3eb0b2e0777152a136f25933254d621"
server_key_iv: "5bbc6abf1bd5319977fdc83dc898cebf"
server_key_version: "encryptionkey@0"
I20260809 18:52:16.975625 6516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:17.008462 6516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:17.010442 6516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:17.010571 6516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:17.010771 6516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:17.011034 6516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:17.011066 6516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:17.011092 6516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:17.011109 6516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:17.016321 6516 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:45557
I20260809 18:52:17.016402 6720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:45557 every 8 connection(s)
I20260809 18:52:17.016675 6516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 18:52:17.023066 6299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:17.017351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:32785 (local address 127.23.218.254:36389)
0809 18:52:17.017401 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:17.017403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:17.017549 (+ 146us) server_negotiation.cc:407] Connection header received
0809 18:52:17.017995 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:17.017996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:17.018004 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:17.018054 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:17.018863 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.019333 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.020215 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.020390 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.020430 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.023011 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:32785: BlockingRecv error: recv got EOF from 127.23.218.195:32785 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 18:52:17.023094 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:17.017305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:32785)
0809 18:52:17.017447 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:17.017484 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:17.017917 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:17.018103 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:17.018106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:17.018614 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:17.018793 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.018798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.019399 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.019400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.020155 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.020158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.020432 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.020432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.020515 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.020922 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:17.020930 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:17.022920 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
W20260809 18:52:17.023365 6722 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:36389 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.218.254:36389: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
I20260809 18:52:17.025936 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 6516
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:17.027431 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:17.023576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:41563)
0809 18:52:17.023611 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:17.023621 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:17.023673 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:17.023898 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:17.023898 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:17.023996 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:17.024034 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.024035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.024561 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.024561 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.025020 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.025021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.025284 (+ 263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.025284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.025328 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.025651 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:17.025657 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:17.027371 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260809 18:52:17.027482 6299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:17.023750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:41563 (local address 127.23.218.254:36389)
0809 18:52:17.023799 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:17.023801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:17.023809 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:17.023830 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:17.023831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:17.023839 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:17.023883 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:17.024084 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.024510 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.025072 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.025230 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.025274 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.027445 (+ 2171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:41563: BlockingRecv error: recv got EOF from 127.23.218.195:41563 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:17.032133 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:17.027783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:51075)
0809 18:52:17.027831 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:17.027841 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:17.027878 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:17.027985 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:17.027986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:17.028115 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:17.028140 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.028141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.028944 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.028945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.029630 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.029633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.030033 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.030034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.030070 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.030427 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:17.030433 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:17.032096 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 18:52:17.032248 6299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:17.027854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:51075 (local address 127.23.218.254:36389)
0809 18:52:17.027902 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:17.027903 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:17.027910 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:17.027928 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:17.027929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:17.027936 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:17.027973 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:17.028200 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.028879 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.029709 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.029961 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.030029 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.032200 (+ 2171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:51075: BlockingRecv error: recv got EOF from 127.23.218.195:51075 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260809 18:52:17.032466 6722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 18:52:17.033414 6304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:17.026308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.23.218.254:36389)
0809 18:52:17.026386 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:17.026388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:17.026487 (+ 99us) server_negotiation.cc:407] Connection header received
0809 18:52:17.026566 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:17.026567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:17.026578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:17.026641 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:17.027310 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.027990 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.028997 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.029299 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.029364 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.030554 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:17.030558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:17.030559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:17.030570 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:17.032380 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:17.032699 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:17.032701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:17.032701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:17.032781 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:17.032903 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:17.032904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:17.032904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:17.033080 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:17.033175 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:17.033246 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 18:52:17.033322 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:17.783514 6729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:17.779346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:53565)
0809 18:52:17.779484 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:17.779495 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:17.779547 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:17.779681 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:17.779682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:17.779834 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:17.779902 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.779904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.780495 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.780496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.781024 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.781025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.781319 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.781319 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.781375 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.781817 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:17.781823 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:17.783460 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
W20260809 18:52:17.783612 6730 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:17.779409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:53565 (local address 127.23.218.254:36389)
0809 18:52:17.779522 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:17.779523 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:17.779533 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:52:17.779598 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:17.779598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:17.779609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:17.779672 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:17.779953 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.780441 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.781091 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.781268 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.781309 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.783546 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:53565: BlockingRecv error: recv got EOF from 127.23.218.193:53565 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":53,"threads_started":1}
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:17 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:17.900417 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:17.895240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:36791)
0809 18:52:17.895343 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:17.895353 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:17.895401 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:17.895543 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:17.895544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:17.895714 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:17.895769 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.895771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.896624 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.896626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.897444 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:17.897447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.897822 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.897823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:17.897875 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.898315 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:17.898319 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:17.900340 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":41,"threads_started":1}
W20260809 18:52:17.900480 6730 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:17.895325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:36791 (local address 127.23.218.254:36389)
0809 18:52:17.895370 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:17.895373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:17.895385 (+ 12us) server_negotiation.cc:407] Connection header received
0809 18:52:17.895455 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:17.895456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:17.895469 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:17.895538 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:17.895843 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.896563 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.897519 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:17.897764 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:17.897822 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:17.900428 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:36791: BlockingRecv error: recv got EOF from 127.23.218.194:36791 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 09 18:52:18 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:18 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:18.037084 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:18.032763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:34957)
0809 18:52:18.032880 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:18.032894 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:18.032961 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:18.033092 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:18.033093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:18.033259 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:18.033306 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:18.033309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.033953 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.033954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:18.034572 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:18.034576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.034916 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.034917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:18.034972 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:18.035360 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:18.035364 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:18.037026 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
W20260809 18:52:18.037199 6730 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:18.032837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:34957 (local address 127.23.218.254:36389)
0809 18:52:18.032913 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:18.032915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:18.032971 (+ 56us) server_negotiation.cc:407] Connection header received
0809 18:52:18.033016 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:18.033017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:18.033031 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:18.033077 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:18.033390 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.033873 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.034655 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.034847 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.034904 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:18.037117 (+ 2213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:34957: BlockingRecv error: recv got EOF from 127.23.218.195:34957 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 18:52:18 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:18 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:18.788013 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:18.783919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:41943)
0809 18:52:18.784036 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:18.784046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:18.784093 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:18.784242 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:18.784243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:18.784388 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:18.784435 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:18.784437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.785024 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.785024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:18.785535 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:18.785536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.785832 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.785833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:18.785869 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:18.786319 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:18.786323 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:18.787962 (+ 1639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
W20260809 18:52:18.788139 6735 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:18.784007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:41943 (local address 127.23.218.254:36389)
0809 18:52:18.784137 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:18.784139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:18.784146 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:18.784166 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:18.784167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:18.784177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:18.784231 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:18.784489 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.784971 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.785592 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.785778 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.785823 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:18.788054 (+ 2231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:41943: BlockingRecv error: recv got EOF from 127.23.218.193:41943 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":51,"threads_started":1}
Aug 09 18:52:18 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:18 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:18.904940 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:18.900815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:56205)
0809 18:52:18.900927 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:18.900938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:18.900986 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:18.901115 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:18.901116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:18.901277 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:18.901326 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:18.901328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.901912 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.901913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:18.902485 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:18.902488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.902823 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.902824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:18.902859 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:18.903292 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:18.903297 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:18.904888 (+ 1591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
W20260809 18:52:18.905040 6735 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:18.900885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:56205 (local address 127.23.218.254:36389)
0809 18:52:18.900952 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:18.900953 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:18.900975 (+ 22us) server_negotiation.cc:407] Connection header received
0809 18:52:18.901039 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:18.901040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:18.901050 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:18.901104 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:18.901377 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.901855 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.902567 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:18.902762 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:18.902816 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:18.904983 (+ 2167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:56205: BlockingRecv error: recv got EOF from 127.23.218.194:56205 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 18:52:19 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:19 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:19.041886 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:19.037477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:59789)
0809 18:52:19.037604 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:19.037615 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:19.037679 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:19.037816 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:19.037817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:19.037983 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:19.038031 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:19.038033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.038730 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.038731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:19.039246 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:19.039248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.039587 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.039587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:19.039653 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:19.040105 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:19.040110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:19.041814 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":55,"threads_started":1}
W20260809 18:52:19.041970 6735 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:19.037586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:59789 (local address 127.23.218.254:36389)
0809 18:52:19.037657 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:19.037659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:19.037668 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:19.037740 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:19.037741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:19.037753 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:19.037796 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:19.038112 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.038652 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.039333 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.039516 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.039570 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:19.041919 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:59789: BlockingRecv error: recv got EOF from 127.23.218.195:59789 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 18:52:19 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:19 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:19.793087 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:19.788411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:60745)
0809 18:52:19.788556 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:19.788567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:19.788639 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:19.788764 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:19.788764 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:19.788915 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:19.788965 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:19.788966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.789660 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.789662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:19.790535 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:19.790538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.790911 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.790912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:19.790952 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:19.791343 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:19.791348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:19.793022 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":59,"threads_started":1}
W20260809 18:52:19.793187 6740 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:19.788512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:60745 (local address 127.23.218.254:36389)
0809 18:52:19.788618 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:19.788619 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:19.788628 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:19.788684 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:19.788684 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:19.788695 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:19.788749 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:19.789015 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.789567 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.790619 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.790848 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.790903 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:19.793122 (+ 2219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:60745: BlockingRecv error: recv got EOF from 127.23.218.193:60745 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":47,"threads_started":1}
Aug 09 18:52:19 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:19 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:19.909723 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:19.905418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:36585)
0809 18:52:19.905545 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:19.905556 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:19.905609 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:19.905739 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:19.905740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:19.905893 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:19.905941 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:19.905943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.906586 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.906587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:19.907121 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:19.907123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.907454 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.907454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:19.907489 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:19.907879 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:19.907884 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:19.909660 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
W20260809 18:52:19.909811 6740 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:19.905533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:36585 (local address 127.23.218.254:36389)
0809 18:52:19.905590 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:19.905591 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:19.905600 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:19.905672 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:19.905673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:19.905683 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:19.905731 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:19.905986 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.906520 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.907200 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:19.907395 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:19.907452 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:19.909749 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:36585: BlockingRecv error: recv got EOF from 127.23.218.194:36585 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 18:52:20 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:20 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:20.046314 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:20.042324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:60697)
0809 18:52:20.042456 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:20.042468 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:20.042525 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:20.042641 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:20.042642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:20.042790 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:20.042836 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:20.042838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.043425 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.043426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:20.043952 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:20.043953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.044228 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.044229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:20.044266 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:20.044635 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:20.044638 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:20.046251 (+ 1613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
W20260809 18:52:20.046423 6740 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:20.042411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:60697 (local address 127.23.218.254:36389)
0809 18:52:20.042476 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:20.042477 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:20.042511 (+ 34us) server_negotiation.cc:407] Connection header received
0809 18:52:20.042574 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:20.042575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:20.042588 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:20.042625 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:20.042885 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.043373 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.044004 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.044182 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.044223 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:20.046358 (+ 2135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:60697: BlockingRecv error: recv got EOF from 127.23.218.195:60697 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 18:52:20 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:20 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:20.797499 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:20.793480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:33713)
0809 18:52:20.793595 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:20.793607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:20.793661 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:20.793776 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:20.793777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:20.793926 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:20.793973 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:20.793975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.794587 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.794588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:20.795103 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:20.795104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.795401 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.795401 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:20.795438 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:20.795839 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:20.795844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:20.797448 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
W20260809 18:52:20.797636 6745 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:20.793531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:33713 (local address 127.23.218.254:36389)
0809 18:52:20.793652 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:20.793654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:20.793661 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:52:20.793704 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:20.793705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:20.793715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:20.793767 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:20.794032 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.794535 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.795176 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.795352 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.795394 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:20.797548 (+ 2154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:33713: BlockingRecv error: recv got EOF from 127.23.218.193:33713 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
Aug 09 18:52:20 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:20 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:20.919313 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:20.910099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:33413)
0809 18:52:20.910235 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:20.910246 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:20.910306 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:20.910444 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:20.910445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:20.910591 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:20.910638 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:20.910639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.911244 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.911245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:20.911772 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:20.911773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.912091 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.912091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:20.912143 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:20.917275 (+ 5132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:20.917281 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:20.919259 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
W20260809 18:52:20.919413 6745 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:20.910189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:33413 (local address 127.23.218.254:36389)
0809 18:52:20.910257 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:20.910259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:20.910289 (+ 30us) server_negotiation.cc:407] Connection header received
0809 18:52:20.910374 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:20.910375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:20.910386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:20.910431 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:20.910689 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.911161 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.911848 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:20.912035 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:20.912085 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:20.919353 (+ 7268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:33413: BlockingRecv error: recv got EOF from 127.23.218.194:33413 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 18:52:21 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:21 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:21.050851 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:21.046732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:49119)
0809 18:52:21.046861 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:21.046872 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:21.046938 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:21.047058 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:21.047058 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:21.047217 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:21.047267 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:21.047268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.047862 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.047863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:21.048374 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:21.048375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.048699 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.048699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:21.048736 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:21.049122 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:21.049126 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:21.050795 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
W20260809 18:52:21.050940 6745 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:21.046787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:49119 (local address 127.23.218.254:36389)
0809 18:52:21.046833 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:21.046835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:21.046939 (+ 104us) server_negotiation.cc:407] Connection header received
0809 18:52:21.046991 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:21.046991 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:21.047004 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:21.047046 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:21.047316 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.047807 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.048447 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.048641 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.048694 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:21.050881 (+ 2187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:49119: BlockingRecv error: recv got EOF from 127.23.218.195:49119 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 18:52:21 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:21 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:21.802088 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:21.797909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:53469)
0809 18:52:21.798022 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:21.798031 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:21.798078 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:21.798223 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:21.798224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:21.798391 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:21.798441 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:21.798443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.799037 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.799038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:21.799556 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:21.799558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.799847 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.799847 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:21.799882 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:21.800269 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:21.800273 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:21.802035 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
W20260809 18:52:21.802282 6751 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:21.797982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:53469 (local address 127.23.218.254:36389)
0809 18:52:21.798109 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:21.798111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:21.798120 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:21.798141 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:21.798142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:21.798152 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:21.798208 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:21.798492 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.798981 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.799615 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.799798 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.799841 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:21.802205 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:53469: BlockingRecv error: recv got EOF from 127.23.218.193:53469 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
Aug 09 18:52:21 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:21 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:21.923959 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:21.919731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:50501)
0809 18:52:21.919869 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:21.919880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:21.919937 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:21.920047 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:21.920047 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:21.920191 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:21.920237 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:21.920239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.920810 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.920811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:21.921331 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:21.921332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.921603 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.921604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:21.921641 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:21.922056 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:21.922060 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:21.923901 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":54,"threads_started":1}
W20260809 18:52:21.924050 6751 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:21.919834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:50501 (local address 127.23.218.254:36389)
0809 18:52:21.919891 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:21.919893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:21.919921 (+ 28us) server_negotiation.cc:407] Connection header received
0809 18:52:21.919987 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:21.919987 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:21.919997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:21.920041 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:21.920284 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.920758 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.921385 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:21.921555 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:21.921596 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:21.923991 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:50501: BlockingRecv error: recv got EOF from 127.23.218.194:50501 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 18:52:22 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:22 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:22.055392 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:22.051247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:50377)
0809 18:52:22.051401 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:22.051410 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:22.051464 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:22.051579 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:22.051580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:22.051738 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:22.051787 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:22.051789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.052404 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.052405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:22.052925 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:22.052926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.053212 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.053213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:22.053287 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:22.053698 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:22.053702 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:22.055316 (+ 1614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
W20260809 18:52:22.055486 6751 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:22.051314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:50377 (local address 127.23.218.254:36389)
0809 18:52:22.051366 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:22.051368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:22.051449 (+ 81us) server_negotiation.cc:407] Connection header received
0809 18:52:22.051505 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:22.051505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:22.051519 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:22.051566 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:22.051837 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.052339 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.052980 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.053153 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.053195 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:22.055416 (+ 2221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:50377: BlockingRecv error: recv got EOF from 127.23.218.195:50377 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 18:52:22 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:22 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:22.806821 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:22.802527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:34743)
0809 18:52:22.802647 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:22.802657 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:22.802708 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:22.802824 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:22.802824 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:22.802975 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:22.803020 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:22.803021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.803630 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.803631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:22.804176 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:22.804178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.804496 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.804497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:22.804540 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:22.804982 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:22.804986 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:22.806769 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
W20260809 18:52:22.806922 6755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:22.802600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:34743 (local address 127.23.218.254:36389)
0809 18:52:22.802699 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:22.802701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:22.802709 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:22.802749 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:22.802750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:22.802759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:22.802813 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:22.803073 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.803570 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.804246 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.804441 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.804486 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:22.806854 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:34743: BlockingRecv error: recv got EOF from 127.23.218.193:34743 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":48,"threads_started":1}
Aug 09 18:52:22 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:22 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:22.928486 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:22.924385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:51221)
0809 18:52:22.924501 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:22.924512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:22.924570 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:22.924689 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:22.924690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:22.924829 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:22.924875 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:22.924877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.925449 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.925449 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:22.925989 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:22.925990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.926266 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.926266 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:22.926303 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:22.926744 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:22.926749 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:22.928431 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
W20260809 18:52:22.928577 6755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:22.924448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:51221 (local address 127.23.218.254:36389)
0809 18:52:22.924529 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:22.924531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:22.924570 (+ 39us) server_negotiation.cc:407] Connection header received
0809 18:52:22.924619 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:22.924620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:22.924630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:22.924677 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:22.924922 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.925398 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.926044 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:22.926218 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:22.926259 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:22.928520 (+ 2261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:51221: BlockingRecv error: recv got EOF from 127.23.218.194:51221 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 18:52:23 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:23 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:23.059736 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:23.055807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:56293)
0809 18:52:23.055941 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:23.055950 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:23.056003 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:23.056119 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:23.056120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:23.056266 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:23.056313 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:23.056314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.056898 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.056898 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:23.057405 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:23.057406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.057683 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.057683 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:23.057720 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:23.058100 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:23.058104 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:23.059682 (+ 1578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
W20260809 18:52:23.059828 6755 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:23.055869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:56293 (local address 127.23.218.254:36389)
0809 18:52:23.055925 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:23.055927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:23.056000 (+ 73us) server_negotiation.cc:407] Connection header received
0809 18:52:23.056054 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:23.056054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:23.056067 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:23.056103 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:23.056361 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.056846 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.057455 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.057626 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.057676 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:23.059771 (+ 2095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:56293: BlockingRecv error: recv got EOF from 127.23.218.195:56293 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 18:52:23 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:23 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:23.811488 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:23.807223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:39981)
0809 18:52:23.807343 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:23.807354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:23.807416 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:23.807572 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:23.807573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:23.807755 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:23.807806 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:23.807808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.808423 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.808424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:23.808967 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:23.808969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.809248 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.809248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:23.809285 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:23.809720 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:23.809724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:23.811414 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
W20260809 18:52:23.811576 6763 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:23.807267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:39981 (local address 127.23.218.254:36389)
0809 18:52:23.807367 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:23.807369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:23.807401 (+ 32us) server_negotiation.cc:407] Connection header received
0809 18:52:23.807475 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:23.807476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:23.807490 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:23.807556 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:23.807867 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.808350 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.809024 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.809200 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.809241 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:23.811516 (+ 2275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:39981: BlockingRecv error: recv got EOF from 127.23.218.193:39981 (error 108)
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":54,"threads_started":1}
Aug 09 18:52:23 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:23 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:23.933284 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:23.928917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:38641)
0809 18:52:23.929052 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:23.929062 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:23.929127 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:23.929246 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:23.929247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:23.929408 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:23.929458 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:23.929460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.930110 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.930111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:23.930707 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:23.930710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.931048 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.931049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:23.931097 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:23.931527 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:23.931532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:23.933229 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
W20260809 18:52:23.933399 6763 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:23.928981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:38641 (local address 127.23.218.254:36389)
0809 18:52:23.929046 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:23.929048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:23.929118 (+ 70us) server_negotiation.cc:407] Connection header received
0809 18:52:23.929182 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:23.929183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:23.929194 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:23.929230 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:23.929509 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.930037 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.930789 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:23.930983 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:23.931029 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:23.933331 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:38641: BlockingRecv error: recv got EOF from 127.23.218.194:38641 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 18:52:24 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:24 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:24.064366 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:24.060159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:40999)
0809 18:52:24.060308 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:24.060317 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:24.060372 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:24.060491 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:24.060492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:24.060637 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:24.060688 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:24.060689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.061478 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.061479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:24.061993 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:24.061995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.062297 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.062298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:24.062380 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:24.062776 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:24.062780 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:24.064310 (+ 1530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
W20260809 18:52:24.064453 6763 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:24.060244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:40999 (local address 127.23.218.254:36389)
0809 18:52:24.060310 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:24.060311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:24.060368 (+ 57us) server_negotiation.cc:407] Connection header received
0809 18:52:24.060414 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:24.060415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:24.060428 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:24.060481 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:24.060737 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.061415 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.062058 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.062248 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.062291 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:24.064396 (+ 2105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:40999: BlockingRecv error: recv got EOF from 127.23.218.195:40999 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 18:52:24 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:24 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:24.816229 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:24.811959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:35047)
0809 18:52:24.812094 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:24.812107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:24.812178 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:24.812321 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:24.812322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:24.812520 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:24.812578 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:24.812581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.813216 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.813216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:24.813733 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:24.813734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.814020 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.814021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:24.814056 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:24.814540 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:24.814546 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:24.816155 (+ 1609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
W20260809 18:52:24.816330 6768 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:24.812062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:35047 (local address 127.23.218.254:36389)
0809 18:52:24.812198 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:24.812200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:24.812209 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:24.812228 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:24.812228 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:24.812240 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:24.812315 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:24.812657 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.813156 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.813792 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.813972 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.814015 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:24.816267 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:35047: BlockingRecv error: recv got EOF from 127.23.218.193:35047 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
Aug 09 18:52:24 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:24 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:24.937809 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:24.933714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:58993)
0809 18:52:24.933849 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:24.933859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:24.933919 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:24.934028 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:24.934028 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:24.934191 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:24.934242 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:24.934243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.934986 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.934987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:24.935516 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:24.935518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.935794 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.935794 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:24.935831 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:24.936209 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:24.936213 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:24.937754 (+ 1541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
W20260809 18:52:24.937899 6768 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:24.933767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:58993 (local address 127.23.218.254:36389)
0809 18:52:24.933826 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:24.933828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:24.933914 (+ 86us) server_negotiation.cc:407] Connection header received
0809 18:52:24.933959 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:24.933960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:24.933971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:24.934017 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:24.934292 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.934922 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.935576 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:24.935748 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:24.935790 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:24.937843 (+ 2053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:58993: BlockingRecv error: recv got EOF from 127.23.218.194:58993 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 18:52:25 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:25 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:25.069212 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:25.064779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:51363)
0809 18:52:25.064896 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:25.064906 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:25.064962 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:25.065089 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:25.065090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:25.065234 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:25.065282 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:25.065284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.065873 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.065874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:25.066492 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:25.066496 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.066869 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.066871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:25.066934 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:25.067336 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:25.067340 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:25.069140 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":53,"threads_started":1}
W20260809 18:52:25.069293 6768 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:25.064844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:51363 (local address 127.23.218.254:36389)
0809 18:52:25.064924 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:25.064925 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:25.064945 (+ 20us) server_negotiation.cc:407] Connection header received
0809 18:52:25.065011 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:25.065011 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:25.065023 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:25.065067 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:25.065331 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.065810 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.066562 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.066796 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.066851 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:25.069241 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:51363: BlockingRecv error: recv got EOF from 127.23.218.195:51363 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 18:52:25 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:25 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:25.821009 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:25.816733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:56691)
0809 18:52:25.816885 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:25.816898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:25.816955 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:25.817120 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:25.817121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:25.817296 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:25.817346 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:25.817348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.817955 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.817955 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:25.818504 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:25.818506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.818799 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.818800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:25.818834 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:25.819231 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:25.819235 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:25.820933 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
W20260809 18:52:25.821113 6776 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:25.816805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:56691 (local address 127.23.218.254:36389)
0809 18:52:25.816933 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:25.816935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:25.816944 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:25.816999 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:25.817000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:25.817011 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:25.817070 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:25.817408 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.817901 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.818557 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.818748 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.818797 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:25.821028 (+ 2231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:56691: BlockingRecv error: recv got EOF from 127.23.218.193:56691 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
Aug 09 18:52:25 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:25 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:25.946242 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:25.938251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:53949)
0809 18:52:25.938402 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:25.938413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:25.938470 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:25.938581 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:25.938581 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:25.938744 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:25.938797 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:25.938798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.939388 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.939389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:25.939907 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:25.939908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.940185 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.940186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:25.940222 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:25.944533 (+ 4311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:25.944539 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:25.946184 (+ 1645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
W20260809 18:52:25.946359 6776 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:25.938288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:53949 (local address 127.23.218.254:36389)
0809 18:52:25.938354 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:25.938356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:25.938453 (+ 97us) server_negotiation.cc:407] Connection header received
0809 18:52:25.938510 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:25.938511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:25.938523 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:25.938569 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:25.938846 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.939337 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.939959 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:25.940130 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:25.940179 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:25.946284 (+ 6105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:53949: BlockingRecv error: recv got EOF from 127.23.218.194:53949 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 18:52:26 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:26 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:26.073949 6778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:26.069639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:36043)
0809 18:52:26.069791 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:26.069801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:26.069859 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:26.069979 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:26.069980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:26.070129 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:26.070180 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:26.070182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.070860 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.070861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:26.071369 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:26.071370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.071667 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.071668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:26.071703 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:26.072104 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:26.072107 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:26.073855 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
W20260809 18:52:26.074009 6776 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:26.069696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:36043 (local address 127.23.218.254:36389)
0809 18:52:26.069749 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:26.069751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:26.069874 (+ 123us) server_negotiation.cc:407] Connection header received
0809 18:52:26.069901 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:26.069901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:26.069915 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:26.069969 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:26.070242 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.070803 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.071424 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.071611 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.071659 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:26.073956 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:36043: BlockingRecv error: recv got EOF from 127.23.218.195:36043 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 18:52:26 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:26 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:26.825734 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:26.821613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:46033)
0809 18:52:26.821738 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:26.821749 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:26.821798 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:26.821934 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:26.821935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:26.822089 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:26.822137 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:26.822139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.822787 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.822788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:26.823308 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:26.823309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.823598 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.823598 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:26.823641 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:26.824104 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:26.824109 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:26.825679 (+ 1570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
W20260809 18:52:26.825826 6782 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:26.821695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:46033 (local address 127.23.218.254:36389)
0809 18:52:26.821825 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:26.821827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:26.821835 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:26.821855 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:26.821856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:26.821866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:26.821924 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:26.822201 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.822724 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.823366 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.823549 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.823594 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:26.825761 (+ 2167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:46033: BlockingRecv error: recv got EOF from 127.23.218.193:46033 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
Aug 09 18:52:26 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:26 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:26.950932 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:26.946683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:33699)
0809 18:52:26.946811 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:26.946822 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:26.946882 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:26.946992 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:26.946993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:26.947159 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:26.947207 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:26.947208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.947822 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.947822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:26.948334 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:26.948336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.948642 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.948642 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:26.948682 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:26.949111 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:26.949116 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:26.950847 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
W20260809 18:52:26.951000 6782 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:26.946747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:33699 (local address 127.23.218.254:36389)
0809 18:52:26.946822 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:26.946824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:26.946866 (+ 42us) server_negotiation.cc:407] Connection header received
0809 18:52:26.946925 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:26.946925 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:26.946937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:26.946980 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:26.947272 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.947767 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.948403 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:26.948586 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:26.948636 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:26.950948 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:33699: BlockingRecv error: recv got EOF from 127.23.218.194:33699 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 18:52:27 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:27 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:27.078688 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:27.074434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:54381)
0809 18:52:27.074547 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:27.074559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:27.074622 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:27.074755 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:27.074755 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:27.074918 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:27.074965 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:27.074966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.075596 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.075597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:27.076113 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:27.076115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.076431 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.076432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:27.076486 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:27.076881 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:27.076884 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:27.078590 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
W20260809 18:52:27.078742 6782 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:27.074518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:54381 (local address 127.23.218.254:36389)
0809 18:52:27.074573 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:27.074576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:27.074603 (+ 27us) server_negotiation.cc:407] Connection header received
0809 18:52:27.074687 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:27.074688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:27.074703 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:27.074748 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:27.075029 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.075526 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.076187 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.076370 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.076424 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:27.078690 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:54381: BlockingRecv error: recv got EOF from 127.23.218.195:54381 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 18:52:27 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:27 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:27.830636 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:27.826120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:36977)
0809 18:52:27.826244 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:27.826253 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:27.826298 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:27.826443 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:27.826443 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:27.826606 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:27.826657 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:27.826658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.827443 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.827444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:27.827977 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:27.827978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.828318 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.828319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:27.828375 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:27.828804 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:27.828808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:27.830542 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
W20260809 18:52:27.830730 6790 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:27.826200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:36977 (local address 127.23.218.254:36389)
0809 18:52:27.826309 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:27.826311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:27.826334 (+ 23us) server_negotiation.cc:407] Connection header received
0809 18:52:27.826375 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:27.826375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:27.826386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:27.826434 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:27.826745 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.827370 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.828050 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.828256 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.828311 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:27.830652 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:36977: BlockingRecv error: recv got EOF from 127.23.218.193:36977 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":52,"threads_started":1}
Aug 09 18:52:27 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:27 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:27.957041 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:27.951372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:59713)
0809 18:52:27.951489 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:27.951500 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:27.951563 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:27.951753 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:27.951754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:27.951957 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:27.952016 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:27.952019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.952966 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.952967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:27.953811 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:27.953815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.954268 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.954269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:27.954332 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:27.954832 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:27.954838 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:27.956951 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
W20260809 18:52:27.957144 6790 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:27.951421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:59713 (local address 127.23.218.254:36389)
0809 18:52:27.951471 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:27.951473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:27.951548 (+ 75us) server_negotiation.cc:407] Connection header received
0809 18:52:27.951624 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:27.951625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:27.951638 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:27.951699 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:27.952102 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.952881 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.953893 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:27.954191 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:27.954258 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:27.957071 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:59713: BlockingRecv error: recv got EOF from 127.23.218.194:59713 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 18:52:28 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:28 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:28.083246 6792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:28.079079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:46919)
0809 18:52:28.079225 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:28.079234 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:28.079285 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:28.079395 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:28.079395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:28.079537 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:28.079582 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:28.079584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.080282 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.080283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:28.080853 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:28.080856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.081170 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.081171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:28.081206 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:28.081605 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:28.081609 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:28.083183 (+ 1574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
W20260809 18:52:28.083312 6790 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:28.079144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:46919 (local address 127.23.218.254:36389)
0809 18:52:28.079187 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:28.079189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:28.079281 (+ 92us) server_negotiation.cc:407] Connection header received
0809 18:52:28.079323 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:28.079324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:28.079336 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:28.079383 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:28.079629 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.080202 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.080917 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.081104 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.081158 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:28.083261 (+ 2103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:46919: BlockingRecv error: recv got EOF from 127.23.218.195:46919 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 18:52:28 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:28 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:28.835229 6793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:28.831096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:46385)
0809 18:52:28.831211 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:28.831221 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:28.831268 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:28.831380 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:28.831381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:28.831540 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:28.831589 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:28.831590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.832212 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.832213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:28.832745 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:28.832747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.833026 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.833027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:28.833062 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:28.833465 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:28.833469 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:28.835164 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":64,"threads_started":1}
W20260809 18:52:28.835320 6794 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:28.831163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:46385 (local address 127.23.218.254:36389)
0809 18:52:28.831274 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:28.831275 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:28.831284 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:28.831304 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:28.831305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:28.831314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:28.831370 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:28.831642 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.832147 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.832802 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.832977 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.833018 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:28.835254 (+ 2236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:46385: BlockingRecv error: recv got EOF from 127.23.218.193:46385 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
Aug 09 18:52:28 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:28 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:28.961794 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:28.957611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:58467)
0809 18:52:28.957767 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:28.957777 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:28.957835 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:28.957949 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:28.957950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:28.958113 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:28.958165 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:28.958166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.958784 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.958784 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:28.959321 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:28.959323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.959612 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.959613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:28.959648 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:28.960085 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:28.960090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:28.961730 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
W20260809 18:52:28.961899 6794 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:28.957677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:58467 (local address 127.23.218.254:36389)
0809 18:52:28.957738 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:28.957741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:28.957831 (+ 90us) server_negotiation.cc:407] Connection header received
0809 18:52:28.957879 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:28.957880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:28.957891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:28.957938 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:28.958215 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.958731 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.959383 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:28.959561 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:28.959606 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:28.961829 (+ 2223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:58467: BlockingRecv error: recv got EOF from 127.23.218.194:58467 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 18:52:29 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:29 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:29.087750 6797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:29.083654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:42041)
0809 18:52:29.083786 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:29.083796 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:29.083851 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:29.083963 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:29.083963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:29.084136 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:29.084183 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:29.084185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.084760 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.084760 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:29.085296 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:29.085298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.085562 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.085562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:29.085597 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:29.086019 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:29.086023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:29.087683 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
W20260809 18:52:29.087823 6794 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:29.083693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:42041 (local address 127.23.218.254:36389)
0809 18:52:29.083736 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:29.083738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:29.083840 (+ 102us) server_negotiation.cc:407] Connection header received
0809 18:52:29.083901 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:29.083901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:29.083913 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:29.083949 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:29.084234 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.084708 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.085348 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.085516 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.085558 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:29.087770 (+ 2212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:42041: BlockingRecv error: recv got EOF from 127.23.218.195:42041 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 18:52:29 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:29 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:29.839839 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:29.835628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:37547)
0809 18:52:29.835777 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:29.835786 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:29.835838 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:29.835965 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:29.835966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:29.836123 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:29.836183 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:29.836185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.836776 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.836776 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:29.837307 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:29.837308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.837604 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.837605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:29.837641 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:29.838040 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:29.838045 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:29.839783 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
W20260809 18:52:29.839982 6800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:29.835681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:37547 (local address 127.23.218.254:36389)
0809 18:52:29.835801 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:29.835803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:29.835823 (+ 20us) server_negotiation.cc:407] Connection header received
0809 18:52:29.835886 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:29.835887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:29.835898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:29.835954 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:29.836233 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.836724 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.837366 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.837554 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.837603 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:29.839903 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:37547: BlockingRecv error: recv got EOF from 127.23.218.193:37547 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
Aug 09 18:52:29 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:29 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:29.966177 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:29.962183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:43677)
0809 18:52:29.962315 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:29.962326 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:29.962409 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:29.962532 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:29.962533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:29.962688 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:29.962737 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:29.962738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.963312 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.963312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:29.963844 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:29.963845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.964113 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.964114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:29.964155 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:29.964521 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:29.964525 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:29.966104 (+ 1579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":56,"threads_started":1}
W20260809 18:52:29.966251 6800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:29.962277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:43677 (local address 127.23.218.254:36389)
0809 18:52:29.962320 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:29.962322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:29.962404 (+ 82us) server_negotiation.cc:407] Connection header received
0809 18:52:29.962465 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:29.962465 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:29.962477 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:29.962513 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:29.962786 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.963258 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.963897 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:29.964069 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:29.964111 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:29.966193 (+ 2082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:43677: BlockingRecv error: recv got EOF from 127.23.218.194:43677 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 18:52:30 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:30 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:30.092432 6802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:30.088171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:34051)
0809 18:52:30.088300 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:30.088310 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:30.088374 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:30.088500 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:30.088501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:30.088679 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:30.088721 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:30.088723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.089362 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.089363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:30.089917 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:30.089919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.090255 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.090256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:30.090310 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:30.090729 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:30.090733 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:30.092375 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
W20260809 18:52:30.092520 6800 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:30.088235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:34051 (local address 127.23.218.254:36389)
0809 18:52:30.088283 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:30.088285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:30.088354 (+ 69us) server_negotiation.cc:407] Connection header received
0809 18:52:30.088430 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:30.088431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:30.088444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:30.088486 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:30.088788 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.089291 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.089989 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.090195 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.090247 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:30.092463 (+ 2216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:34051: BlockingRecv error: recv got EOF from 127.23.218.195:34051 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 18:52:30 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:30 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:30.844280 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:30.840234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:43781)
0809 18:52:30.840358 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:30.840368 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:30.840416 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:30.840533 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:30.840534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:30.840698 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:30.840745 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:30.840746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.841411 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.841412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:30.841945 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:30.841946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.842217 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.842218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:30.842255 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:30.842699 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:30.842704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:30.844224 (+ 1520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
W20260809 18:52:30.844378 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:30.840303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:43781 (local address 127.23.218.254:36389)
0809 18:52:30.840409 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:30.840410 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:30.840418 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:30.840458 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:30.840459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:30.840469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:30.840523 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:30.840821 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.841344 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.841993 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.842173 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.842216 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:30.844307 (+ 2091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:43781: BlockingRecv error: recv got EOF from 127.23.218.193:43781 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":50,"threads_started":1}
Aug 09 18:52:30 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:30 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:30.975626 6807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:30.966642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:54701)
0809 18:52:30.966779 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:30.966792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:30.966860 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:30.967016 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:30.967017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:30.967203 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:30.967253 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:30.967255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.967856 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.967857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:30.968369 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:30.968370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.968660 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.968660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:30.968695 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:30.973776 (+ 5081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:30.973781 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:30.975552 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
W20260809 18:52:30.975709 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:30.966744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:54701 (local address 127.23.218.254:36389)
0809 18:52:30.966807 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:30.966809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:30.966841 (+ 32us) server_negotiation.cc:407] Connection header received
0809 18:52:30.966925 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:30.966926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:30.966938 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:30.967000 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:30.967313 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.967801 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.968425 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:30.968610 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:30.968658 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:30.975657 (+ 6999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:54701: BlockingRecv error: recv got EOF from 127.23.218.194:54701 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 18:52:31 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:31 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.195@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:31.097200 6808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:31.092871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.195:47061)
0809 18:52:31.093024 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:31.093035 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:31.093098 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:31.093221 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:31.093221 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:31.093380 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:31.093429 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:31.093430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.094199 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.094200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:31.094785 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:31.094788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.095081 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.095081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:31.095119 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:31.095518 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:31.095522 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:31.097143 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
W20260809 18:52:31.097285 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:31.092932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:47061 (local address 127.23.218.254:36389)
0809 18:52:31.092992 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:31.092994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:31.093094 (+ 100us) server_negotiation.cc:407] Connection header received
0809 18:52:31.093152 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:31.093153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:31.093167 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:31.093206 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:31.093477 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.094097 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.094846 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.095026 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.095074 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:31.097227 (+ 2153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.195:47061: BlockingRecv error: recv got EOF from 127.23.218.195:47061 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 18:52:31 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:31 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.193@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:31.849026 6809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:31.844756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.193:48039)
0809 18:52:31.844898 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:31.844908 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:31.844960 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:31.845088 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:31.845089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:31.845260 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:31.845309 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:31.845311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.846048 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.846049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:31.846613 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:31.846615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.846903 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.846904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:31.846939 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:31.847342 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:31.847347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:31.848970 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260809 18:52:31.849124 6810 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:31.844840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:48039 (local address 127.23.218.254:36389)
0809 18:52:31.844943 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:31.844944 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:31.844953 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:31.845014 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:31.845015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:31.845026 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:31.845071 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:31.845371 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.845977 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.846670 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.846857 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.846901 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:31.849056 (+ 2155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.193:48039: BlockingRecv error: recv got EOF from 127.23.218.193:48039 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":52,"threads_started":1}
Aug 09 18:52:31 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 18:52:31 dist-test-slave-6zbq krb5kdc[5133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.218.194@KRBTEST.COM for oryx/127.23.218.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 18:52:31.980350 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:31.976094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36389 (local address 127.23.218.194:55499)
0809 18:52:31.976229 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:31.976240 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:31.976298 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:31.976407 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:31.976408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:31.976559 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:31.976610 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:31.976611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.977273 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.977274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:31.977845 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:31.977846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.978183 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.978184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:31.978222 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:31.978617 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:31.978622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:31.980284 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.218.254:36389: Server oryx/127.23.218.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
W20260809 18:52:31.980427 6810 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:31.976166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:55499 (local address 127.23.218.254:36389)
0809 18:52:31.976231 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:31.976233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:31.976282 (+ 49us) server_negotiation.cc:407] Connection header received
0809 18:52:31.976341 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:31.976342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:31.976352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:31.976397 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:31.976661 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.977200 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.977903 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:31.978109 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:31.978186 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:31.980370 (+ 2184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.218.194:55499: BlockingRecv error: recv got EOF from 127.23.218.194:55499 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 18:52:32.035392 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 6070
I20260809 18:52:32.047875 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 6300
I20260809 18:52:32.067976 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 6516
I20260809 18:52:32.074380 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 5964
2026-08-09T18:52:32Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17543 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T18:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:32Z Disabled control of system clock
I20260809 18:52:32.103104 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:33461
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:33461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 18:52:32.180397 6818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:32.180572 6818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:32.180591 6818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:32.181740 6818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:32.181778 6818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:32.181790 6818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:32.181802 6818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:32.181818 6818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:32.183109 6818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:33461
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:33461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:32.183284 6818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:32.183498 6818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:32.187850 6831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.187911 6832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.187916 6836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:32.187971 6818 server_base.cc:1034] running on GCE node
I20260809 18:52:32.188246 6818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:32.188505 6818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:32.189672 6818 hybrid_clock.cc:648] HybridClock initialized: now 1786301552189658 us; error 33 us; skew 500 ppm
I20260809 18:52:32.190829 6818 webserver.cc:460] Webserver started at http://127.23.218.254:44769/ using document root <none> and password file <none>
I20260809 18:52:32.191030 6818 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:32.191073 6818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:32.191198 6818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:32.192153 6818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "067d4ccffcd4407aa71d9a8389dd2810"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "af425ce434de2cca5181e0e57370ff71"
server_key_iv: "477251cea22d81bf07abbface6bc0851"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.192467 6818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "067d4ccffcd4407aa71d9a8389dd2810"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "af425ce434de2cca5181e0e57370ff71"
server_key_iv: "477251cea22d81bf07abbface6bc0851"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.193606 6818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:52:32.194247 6844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.194408 6818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.194485 6818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "067d4ccffcd4407aa71d9a8389dd2810"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "af425ce434de2cca5181e0e57370ff71"
server_key_iv: "477251cea22d81bf07abbface6bc0851"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.194552 6818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:32.207512 6818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:32.207744 6818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:32.207858 6818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:32.211447 6818 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:33461
I20260809 18:52:32.211476 6926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:33461 every 8 connection(s)
I20260809 18:52:32.211742 6818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:32.212309 6928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:32.214691 6928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810: Bootstrap starting.
I20260809 18:52:32.215217 6928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:32.215462 6928 log.cc:824] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:32.216042 6928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810: No bootstrap required, opened a new log
I20260809 18:52:32.216811 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 6818
I20260809 18:52:32.216894 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:32.217046 24427 external_mini_cluster.cc:1428] Setting key 856876ce1ef406e07babcacf595ad55b
I20260809 18:52:32.217759 6928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067d4ccffcd4407aa71d9a8389dd2810" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33461 } }
I20260809 18:52:32.217886 6928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:32.217917 6928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 067d4ccffcd4407aa71d9a8389dd2810, State: Initialized, Role: FOLLOWER
I20260809 18:52:32.218052 6928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067d4ccffcd4407aa71d9a8389dd2810" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33461 } }
I20260809 18:52:32.218122 6928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:32.218150 6928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:32.218199 6928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:32.218729 6928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067d4ccffcd4407aa71d9a8389dd2810" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33461 } }
I20260809 18:52:32.218813 6928 leader_election.cc:302] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 067d4ccffcd4407aa71d9a8389dd2810; no voters:
I20260809 18:52:32.218957 6928 leader_election.cc:288] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:32.218982 6939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:32.219076 6939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [term 1 LEADER]: Becoming Leader. State: Replica: 067d4ccffcd4407aa71d9a8389dd2810, State: Running, Role: LEADER
I20260809 18:52:32.219194 6928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:32.219223 6939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067d4ccffcd4407aa71d9a8389dd2810" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33461 } }
I20260809 18:52:32.219579 6940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "067d4ccffcd4407aa71d9a8389dd2810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067d4ccffcd4407aa71d9a8389dd2810" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33461 } } }
I20260809 18:52:32.219741 6940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:32.219590 6941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 067d4ccffcd4407aa71d9a8389dd2810. Latest consensus state: current_term: 1 leader_uuid: "067d4ccffcd4407aa71d9a8389dd2810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "067d4ccffcd4407aa71d9a8389dd2810" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33461 } } }
I20260809 18:52:32.219991 6941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:32.220162 6950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:32.220681 6950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:32.222611 6936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.217491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44950 (local address 127.23.218.254:33461)
0809 18:52:32.217681 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:32.217688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:32.217710 (+ 22us) server_negotiation.cc:407] Connection header received
0809 18:52:32.217796 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:32.217801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:32.217848 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:32.217959 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:32.218165 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.219582 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.220621 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.221211 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.221332 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:32.221905 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:32.221909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:32.221910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:32.222097 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:32.222136 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:32.222150 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:32.222207 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:32.222311 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 18:52:32.222408 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260809 18:52:32.222645 6950 catalog_manager.cc:1347] Generated new cluster ID: 6a044a574d55479bb45b0837d1886577
I20260809 18:52:32.222687 6950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:32.244179 6950 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:32.244839 6950 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:32.259975 6950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 067d4ccffcd4407aa71d9a8389dd2810: Generated new TSK 0
I20260809 18:52:32.260138 6950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:52:32.267254 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33461
--builtin_ntp_servers=127.23.218.212:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 18:52:32.346626 6964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:32.346812 6964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:32.346832 6964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:32.347942 6964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:32.347981 6964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:32.348004 6964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:32.349308 6964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:32.349547 6964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:32.349817 6964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:32.354375 6969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.354362 6972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.354414 6970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:32.354565 6964 server_base.cc:1034] running on GCE node
I20260809 18:52:32.354746 6964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:32.354959 6964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:32.356083 6964 hybrid_clock.cc:648] HybridClock initialized: now 1786301552356061 us; error 37 us; skew 500 ppm
I20260809 18:52:32.357161 6964 webserver.cc:460] Webserver started at http://127.23.218.193:33715/ using document root <none> and password file <none>
I20260809 18:52:32.357390 6964 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:32.357434 6964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:32.357558 6964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:32.358573 6964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "cd86faf1ba1e4ac1a1ee2d027a051e4c"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "d08c6cc00874b6bb5e4d84483b226d7d"
server_key_iv: "b8c75bb84109700a8b51c1ef86adc316"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.358872 6964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "cd86faf1ba1e4ac1a1ee2d027a051e4c"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "d08c6cc00874b6bb5e4d84483b226d7d"
server_key_iv: "b8c75bb84109700a8b51c1ef86adc316"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.359992 6964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 18:52:32.360626 6979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.360773 6964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:32.360852 6964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "cd86faf1ba1e4ac1a1ee2d027a051e4c"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "d08c6cc00874b6bb5e4d84483b226d7d"
server_key_iv: "b8c75bb84109700a8b51c1ef86adc316"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.360918 6964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:32.381299 6964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:32.381526 6964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:32.381640 6964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:32.381825 6964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:32.382108 6964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:32.382141 6964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.382167 6964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:32.382184 6964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.387341 6964 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:46731
I20260809 18:52:32.387406 7145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:46731 every 8 connection(s)
I20260809 18:52:32.387646 6964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:52:32.392010 6936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.388285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:38203 (local address 127.23.218.254:33461)
0809 18:52:32.388346 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:32.388348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:32.388531 (+ 183us) server_negotiation.cc:407] Connection header received
0809 18:52:32.388958 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:32.388959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:32.388967 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:32.389018 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:32.389576 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.390048 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.390956 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.391155 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.391201 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:32.391814 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:32.391815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:32.391815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:32.391819 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:32.391833 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:32.391836 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:32.391874 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:32.391930 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 18:52:32.391954 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 18:52:32.392165 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.388245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:33461 (local address 127.23.218.193:38203)
0809 18:52:32.388423 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:32.388459 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:32.388888 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:32.389068 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:32.389071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:32.389078 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:32.389517 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:32.389520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.390111 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.390111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:32.390886 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:32.390888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.391209 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.391210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:32.391308 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:32.391703 (+ 395us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:32.391703 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:32.391728 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:32.391748 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:32.391879 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:32.391881 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:32.391884 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:32.391926 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 18:52:32.391997 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260809 18:52:32.392393 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 6964
I20260809 18:52:32.392462 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:52:32.392598 24427 external_mini_cluster.cc:1428] Setting key faa646ea225e9c917467ae6211084757
I20260809 18:52:32.393071 7147 heartbeater.cc:347] Connected to a master server at 127.23.218.254:33461
I20260809 18:52:32.393160 7147 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:32.393322 7147 heartbeater.cc:511] Master 127.23.218.254:33461 requested a full tablet report, sending...
I20260809 18:52:32.393438 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33461
--builtin_ntp_servers=127.23.218.212:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 18:52:32.393836 6871 ts_manager.cc:194] Registered new tserver with Master: cd86faf1ba1e4ac1a1ee2d027a051e4c (127.23.218.193:46731)
I20260809 18:52:32.394606 6871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.193:38203
W20260809 18:52:32.475313 7153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:32.475620 7153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:32.475680 7153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:32.477136 7153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:32.477176 7153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:32.477200 7153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:32.478613 7153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:32.478828 7153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:32.479080 7153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:32.483733 7164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.483757 7165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.483742 7169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:32.483866 7153 server_base.cc:1034] running on GCE node
I20260809 18:52:32.484189 7153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:32.484409 7153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:32.485538 7153 hybrid_clock.cc:648] HybridClock initialized: now 1786301552485526 us; error 29 us; skew 500 ppm
I20260809 18:52:32.486711 7153 webserver.cc:460] Webserver started at http://127.23.218.194:42211/ using document root <none> and password file <none>
I20260809 18:52:32.486914 7153 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:32.486958 7153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:32.487071 7153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:32.488029 7153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "4c0216ed8546488287768acf1e8bdb6e"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "057bb2338ddeaf72aaafd4268310852b"
server_key_iv: "7a5820e02a7f6339e8e7ea36bf531ddc"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.488332 7153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "4c0216ed8546488287768acf1e8bdb6e"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "057bb2338ddeaf72aaafd4268310852b"
server_key_iv: "7a5820e02a7f6339e8e7ea36bf531ddc"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.489446 7153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:32.490048 7183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.490206 7153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:32.490293 7153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "4c0216ed8546488287768acf1e8bdb6e"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "057bb2338ddeaf72aaafd4268310852b"
server_key_iv: "7a5820e02a7f6339e8e7ea36bf531ddc"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.490368 7153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:32.510811 7153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:32.511024 7153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:32.511112 7153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:32.511268 7153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:32.511554 7153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:32.511593 7153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.511619 7153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:32.511637 7153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.516741 7153 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:34167
I20260809 18:52:32.516798 7358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:34167 every 8 connection(s)
I20260809 18:52:32.517068 7153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:52:32.519644 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 7153
I20260809 18:52:32.519747 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:52:32.519918 24427 external_mini_cluster.cc:1428] Setting key 2f519819a7f485588085fe0ca93aaf01
I20260809 18:52:32.520857 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33461
--builtin_ntp_servers=127.23.218.212:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 18:52:32.522259 6936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.517717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:58631 (local address 127.23.218.254:33461)
0809 18:52:32.517778 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:32.517781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:32.517999 (+ 218us) server_negotiation.cc:407] Connection header received
0809 18:52:32.518470 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:32.518471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:32.518483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:32.518537 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:32.519207 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.519762 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.521023 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.521236 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.521266 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:32.522017 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:32.522018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:32.522019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:32.522023 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:32.522039 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:32.522043 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:32.522089 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:32.522155 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 18:52:32.522183 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 18:52:32.522490 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.517681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:33461 (local address 127.23.218.194:58631)
0809 18:52:32.517879 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:32.517916 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:32.518395 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:32.518636 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:32.518639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:32.518649 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:32.519117 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:32.519121 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.519858 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.519859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:32.520922 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:32.520926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.521343 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.521344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:32.521446 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:32.521892 (+ 446us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:32.521893 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:32.521919 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:32.521940 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:32.522091 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:32.522096 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:32.522099 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:32.522158 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 18:52:32.522252 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260809 18:52:32.522998 7360 heartbeater.cc:347] Connected to a master server at 127.23.218.254:33461
I20260809 18:52:32.523087 7360 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:32.523257 7360 heartbeater.cc:511] Master 127.23.218.254:33461 requested a full tablet report, sending...
I20260809 18:52:32.523600 6871 ts_manager.cc:194] Registered new tserver with Master: 4c0216ed8546488287768acf1e8bdb6e (127.23.218.194:34167)
I20260809 18:52:32.523957 6871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.194:58631
W20260809 18:52:32.600677 7364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:32.600857 7364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:32.600886 7364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:32.601980 7364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:32.602017 7364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:32.602051 7364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:32.603381 7364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:32.603592 7364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:32.603893 7364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:32.608711 7369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.608817 7370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.608918 7375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:32.610036 7364 server_base.cc:1034] running on GCE node
I20260809 18:52:32.610282 7364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:32.610541 7364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:32.611650 7364 hybrid_clock.cc:648] HybridClock initialized: now 1786301552611625 us; error 40 us; skew 500 ppm
I20260809 18:52:32.612900 7364 webserver.cc:460] Webserver started at http://127.23.218.195:37687/ using document root <none> and password file <none>
I20260809 18:52:32.613102 7364 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:32.613149 7364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:32.613270 7364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:32.614223 7364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "23a3e76967b749b99b54c02526a3229a"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "82e8ea555bb2736c95eec87c35a8f6c3"
server_key_iv: "2fdc376dce58c7bc27098ec58189f354"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.614574 7364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "23a3e76967b749b99b54c02526a3229a"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "82e8ea555bb2736c95eec87c35a8f6c3"
server_key_iv: "2fdc376dce58c7bc27098ec58189f354"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.615679 7364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:32.616318 7383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.616448 7364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.616528 7364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "23a3e76967b749b99b54c02526a3229a"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "82e8ea555bb2736c95eec87c35a8f6c3"
server_key_iv: "2fdc376dce58c7bc27098ec58189f354"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.616588 7364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:32.623384 7364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:32.623596 7364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:32.623705 7364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:32.623871 7364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:32.624142 7364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:32.624174 7364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.624198 7364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:32.624223 7364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.629441 7364 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:45427
I20260809 18:52:32.629501 7555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:45427 every 8 connection(s)
I20260809 18:52:32.629742 7364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:52:32.634006 6936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.630312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:37347 (local address 127.23.218.254:33461)
0809 18:52:32.630377 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:32.630379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:32.630607 (+ 228us) server_negotiation.cc:407] Connection header received
0809 18:52:32.631022 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:32.631024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:32.631033 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:32.631079 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:32.631639 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.632129 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.632978 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.633152 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.633194 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:32.633813 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:32.633815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:32.633815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:32.633819 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:32.633832 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:32.633836 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:32.633872 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:32.633929 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 18:52:32.633951 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:52:32.634186 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.630306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:33461 (local address 127.23.218.195:37347)
0809 18:52:32.630518 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:32.630547 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:32.630950 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:32.631129 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:32.631138 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:32.631146 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:32.631567 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:32.631570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.632190 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.632190 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:32.632918 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:32.632920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:32.633211 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:32.633211 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:32.633307 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:32.633700 (+ 393us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:32.633700 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:32.633725 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:32.633744 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:32.633879 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:32.633881 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:32.633883 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:52:32.633925 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 18:52:32.633996 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":124,"threads_started":1}
I20260809 18:52:32.634611 7556 heartbeater.cc:347] Connected to a master server at 127.23.218.254:33461
I20260809 18:52:32.634701 7556 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:32.634833 7556 heartbeater.cc:511] Master 127.23.218.254:33461 requested a full tablet report, sending...
I20260809 18:52:32.635118 6871 ts_manager.cc:194] Registered new tserver with Master: 23a3e76967b749b99b54c02526a3229a (127.23.218.195:45427)
I20260809 18:52:32.635461 6871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.195:37347
I20260809 18:52:32.636375 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 7364
I20260809 18:52:32.636445 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:52:32.636581 24427 external_mini_cluster.cc:1428] Setting key a8c2c07f71985946bfc4e2561f82dce9
I20260809 18:52:32.637342 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 18:52:32.638864 6936 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:32.638469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44956 (local address 127.23.218.254:33461)
0809 18:52:32.638512 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:32.638514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:32.638613 (+ 99us) server_negotiation.cc:407] Connection header received
0809 18:52:32.638662 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:32.638663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:32.638670 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:32.638714 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:32.638814 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44956: BlockingRecv error: recv got EOF from 127.0.0.1:44956 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:52:32.638960 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 6964
I20260809 18:52:32.658655 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 7153
I20260809 18:52:32.665419 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 7364
I20260809 18:52:32.672310 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 6818
2026-08-09T18:52:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (601 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T18:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:32Z Disabled control of system clock
I20260809 18:52:32.704556 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:35555
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:35555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 18:52:32.781913 7579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:32.782086 7579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:32.782106 7579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:32.783215 7579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:32.783253 7579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:32.783264 7579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:32.783275 7579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:32.783298 7579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:32.784602 7579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:35555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:35555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:32.784806 7579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:32.785054 7579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:32.789381 7590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.789379 7587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.789412 7588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:32.789577 7579 server_base.cc:1034] running on GCE node
I20260809 18:52:32.789808 7579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:32.790058 7579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:32.791188 7579 hybrid_clock.cc:648] HybridClock initialized: now 1786301552791172 us; error 28 us; skew 500 ppm
I20260809 18:52:32.792245 7579 webserver.cc:460] Webserver started at http://127.23.218.254:41513/ using document root <none> and password file <none>
I20260809 18:52:32.792457 7579 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:32.792503 7579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:32.792618 7579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:32.793545 7579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "a4f250009c074669adcbf4ea93ada4e4"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "5398b683246fa13dbe60a2572de01a51"
server_key_iv: "9a1aa8bc0998daa95212a44afb2412d4"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.793843 7579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "a4f250009c074669adcbf4ea93ada4e4"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "5398b683246fa13dbe60a2572de01a51"
server_key_iv: "9a1aa8bc0998daa95212a44afb2412d4"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.795027 7579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:52:32.795650 7596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.795794 7579 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:32.795876 7579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "a4f250009c074669adcbf4ea93ada4e4"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "5398b683246fa13dbe60a2572de01a51"
server_key_iv: "9a1aa8bc0998daa95212a44afb2412d4"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.795933 7579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:32.799041 7579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:32.799268 7579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:32.799379 7579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:32.802874 7579 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:35555
I20260809 18:52:32.802903 7677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:35555 every 8 connection(s)
I20260809 18:52:32.803181 7579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:32.803978 7679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:32.805882 7679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4: Bootstrap starting.
I20260809 18:52:32.806381 7679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:32.806622 7679 log.cc:824] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:32.807178 7679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4: No bootstrap required, opened a new log
I20260809 18:52:32.808336 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 7579
I20260809 18:52:32.808427 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:32.808590 24427 external_mini_cluster.cc:1428] Setting key 79b29ca90e458b17944a887d07ca307b
I20260809 18:52:32.809026 7679 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f250009c074669adcbf4ea93ada4e4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35555 } }
I20260809 18:52:32.809185 7679 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:32.809228 7679 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f250009c074669adcbf4ea93ada4e4, State: Initialized, Role: FOLLOWER
I20260809 18:52:32.809343 7679 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f250009c074669adcbf4ea93ada4e4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35555 } }
I20260809 18:52:32.809415 7679 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:32.809450 7679 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:32.809494 7679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:32.810079 7679 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f250009c074669adcbf4ea93ada4e4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35555 } }
I20260809 18:52:32.810181 7679 leader_election.cc:302] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4f250009c074669adcbf4ea93ada4e4; no voters:
I20260809 18:52:32.810330 7679 leader_election.cc:288] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:32.810386 7688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:32.810523 7688 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [term 1 LEADER]: Becoming Leader. State: Replica: a4f250009c074669adcbf4ea93ada4e4, State: Running, Role: LEADER
I20260809 18:52:32.810592 7679 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:32.810611 7688 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f250009c074669adcbf4ea93ada4e4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35555 } }
I20260809 18:52:32.810776 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.809017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47506 (local address 127.23.218.254:35555)
0809 18:52:32.809241 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:32.809249 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:32.809278 (+ 29us) server_negotiation.cc:407] Connection header received
0809 18:52:32.809378 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:32.809384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:32.809432 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:32.809564 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:32.810105 (+ 541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:32.810109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:32.810109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:32.810113 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:32.810168 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:32.810182 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:32.810253 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:32.810317 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 18:52:32.810499 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260809 18:52:32.810909 7689 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4f250009c074669adcbf4ea93ada4e4. Latest consensus state: current_term: 1 leader_uuid: "a4f250009c074669adcbf4ea93ada4e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f250009c074669adcbf4ea93ada4e4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35555 } } }
I20260809 18:52:32.810981 7689 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:32.811167 7700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:32.811654 7700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:32.811825 7690 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4f250009c074669adcbf4ea93ada4e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f250009c074669adcbf4ea93ada4e4" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35555 } } }
I20260809 18:52:32.812076 7690 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:32.813126 7700 catalog_manager.cc:1347] Generated new cluster ID: 27179e4434574555b94024ea52a4526b
I20260809 18:52:32.813170 7700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:32.825659 7700 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:32.825783 7700 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:32.830574 7700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4f250009c074669adcbf4ea93ada4e4: Generated new TSK 0
I20260809 18:52:32.830734 7700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:52:32.837553 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:35555
--builtin_ntp_servers=127.23.218.212:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 18:52:32.916512 7720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:32.916708 7720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:32.916741 7720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:32.917959 7720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:32.917996 7720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:32.918030 7720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:32.919407 7720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:35555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:32.919631 7720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:32.919899 7720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:32.925493 7739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.925499 7735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:32.925712 7737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:32.925733 7720 server_base.cc:1034] running on GCE node
I20260809 18:52:32.926004 7720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:32.926257 7720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:32.927423 7720 hybrid_clock.cc:648] HybridClock initialized: now 1786301552927402 us; error 35 us; skew 500 ppm
I20260809 18:52:32.928802 7720 webserver.cc:460] Webserver started at http://127.23.218.193:42263/ using document root <none> and password file <none>
I20260809 18:52:32.929056 7720 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:32.929111 7720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:32.929229 7720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:32.930546 7720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "36eec898ed114de5abac2976978a42b9"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "bf97cb566a6f66e9d7f4f77bdaf8f05c"
server_key_iv: "93cdd330afc7bcf0661b96bceb20498b"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.930954 7720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "36eec898ed114de5abac2976978a42b9"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "bf97cb566a6f66e9d7f4f77bdaf8f05c"
server_key_iv: "93cdd330afc7bcf0661b96bceb20498b"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.932159 7720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:32.932796 7748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.932914 7720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:32.932998 7720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "36eec898ed114de5abac2976978a42b9"
format_stamp: "Formatted at 2026-08-09 18:52:32 on dist-test-slave-6zbq"
server_key: "bf97cb566a6f66e9d7f4f77bdaf8f05c"
server_key_iv: "93cdd330afc7bcf0661b96bceb20498b"
server_key_version: "encryptionkey@0"
I20260809 18:52:32.933068 7720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:32.936261 7720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:32.936484 7720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:32.936595 7720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:32.936781 7720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:32.937049 7720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:32.937080 7720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.937108 7720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:32.937130 7720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:32.942545 7720 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:39235
I20260809 18:52:32.942608 7903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:39235 every 8 connection(s)
I20260809 18:52:32.942889 7720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:52:32.945492 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.943560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:55909 (local address 127.23.218.254:35555)
0809 18:52:32.943639 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:32.943642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:32.943822 (+ 180us) server_negotiation.cc:407] Connection header received
0809 18:52:32.944242 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:32.944243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:32.944251 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:32.944300 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:32.945331 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:32.945331 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:32.945332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:32.945334 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:32.945344 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:32.945347 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:32.945378 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:32.945431 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 18:52:32.945441 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 18:52:32.945667 7908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:32.943532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:35555 (local address 127.23.218.193:55909)
0809 18:52:32.943705 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:32.943744 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:32.944169 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:32.944354 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:32.944357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:32.944365 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:32.945214 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:32.945215 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:32.945243 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:32.945272 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:32.945383 (+ 111us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:32.945385 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:32.945387 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:52:32.945432 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 18:52:32.945477 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260809 18:52:32.946092 7904 heartbeater.cc:347] Connected to a master server at 127.23.218.254:35555
I20260809 18:52:32.946174 7904 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:32.946309 7904 heartbeater.cc:511] Master 127.23.218.254:35555 requested a full tablet report, sending...
I20260809 18:52:32.946768 7620 ts_manager.cc:194] Registered new tserver with Master: 36eec898ed114de5abac2976978a42b9 (127.23.218.193:39235)
I20260809 18:52:32.952653 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 7720
I20260809 18:52:32.952731 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:52:32.952880 24427 external_mini_cluster.cc:1428] Setting key 95bde17c40454cc3fddedd51f0d2da76
I20260809 18:52:32.953717 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:35555
--builtin_ntp_servers=127.23.218.212:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 18:52:33.032436 7909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:33.032609 7909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:33.032637 7909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:33.033713 7909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:33.033746 7909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:33.033769 7909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:33.035080 7909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:35555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:33.035280 7909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:33.035545 7909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:33.040077 7918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:33.040163 7921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:33.040282 7909 server_base.cc:1034] running on GCE node
W20260809 18:52:33.040086 7919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:33.040537 7909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:33.040752 7909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:33.041916 7909 hybrid_clock.cc:648] HybridClock initialized: now 1786301553041902 us; error 27 us; skew 500 ppm
I20260809 18:52:33.043190 7909 webserver.cc:460] Webserver started at http://127.23.218.194:34649/ using document root <none> and password file <none>
I20260809 18:52:33.043403 7909 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:33.043447 7909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:33.043578 7909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:33.044524 7909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "837a29041ed649f3b059ed7b9e0aa446"
format_stamp: "Formatted at 2026-08-09 18:52:33 on dist-test-slave-6zbq"
server_key: "123641109245d1cb94fce65e655f96a1"
server_key_iv: "3cd9bdc21253383e3eb8ee107fb94586"
server_key_version: "encryptionkey@0"
I20260809 18:52:33.044836 7909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "837a29041ed649f3b059ed7b9e0aa446"
format_stamp: "Formatted at 2026-08-09 18:52:33 on dist-test-slave-6zbq"
server_key: "123641109245d1cb94fce65e655f96a1"
server_key_iv: "3cd9bdc21253383e3eb8ee107fb94586"
server_key_version: "encryptionkey@0"
I20260809 18:52:33.045939 7909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:52:33.046577 7931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:33.046700 7909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:33.046773 7909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "837a29041ed649f3b059ed7b9e0aa446"
format_stamp: "Formatted at 2026-08-09 18:52:33 on dist-test-slave-6zbq"
server_key: "123641109245d1cb94fce65e655f96a1"
server_key_iv: "3cd9bdc21253383e3eb8ee107fb94586"
server_key_version: "encryptionkey@0"
I20260809 18:52:33.046837 7909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:33.049877 7909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:33.050096 7909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:33.050201 7909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:33.050413 7909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:33.050685 7909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:33.050717 7909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:33.050743 7909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:33.050766 7909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:33.055953 7909 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:39097
I20260809 18:52:33.055997 8101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:39097 every 8 connection(s)
I20260809 18:52:33.056246 7909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:52:33.058447 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 7909
I20260809 18:52:33.058527 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:52:33.058722 24427 external_mini_cluster.cc:1428] Setting key 381c6b3ab86ffbe1bed6cc744f75bc8b
I20260809 18:52:33.058770 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:33.056834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:37111 (local address 127.23.218.254:35555)
0809 18:52:33.056876 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:33.056878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:33.057076 (+ 198us) server_negotiation.cc:407] Connection header received
0809 18:52:33.057453 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:33.057454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:33.057458 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:33.057506 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:33.058549 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:33.058550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:33.058550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:33.058555 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:33.058573 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:33.058578 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:33.058631 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:33.058690 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 18:52:33.058705 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:52:33.058938 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:33.056814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:35555 (local address 127.23.218.194:37111)
0809 18:52:33.056976 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:33.057010 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:33.057385 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:33.057555 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:33.057558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:33.057566 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:33.058403 (+ 837us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:33.058404 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:33.058429 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:33.058456 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:33.058629 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:33.058639 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:33.058642 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:33.058698 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 18:52:33.058745 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260809 18:52:33.059404 8102 heartbeater.cc:347] Connected to a master server at 127.23.218.254:35555
I20260809 18:52:33.059487 8102 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:33.059613 8102 heartbeater.cc:511] Master 127.23.218.254:35555 requested a full tablet report, sending...
I20260809 18:52:33.059635 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:35555
--builtin_ntp_servers=127.23.218.212:42839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 18:52:33.059986 7619 ts_manager.cc:194] Registered new tserver with Master: 837a29041ed649f3b059ed7b9e0aa446 (127.23.218.194:39097)
W20260809 18:52:33.137023 8105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:33.137202 8105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:33.137223 8105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:33.138281 8105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:33.138312 8105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:33.138335 8105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:33.139638 8105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:35555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:33.139811 8105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:33.140025 8105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 18:52:33.144737 8105 server_base.cc:1034] running on GCE node
W20260809 18:52:33.144672 8114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:33.144678 8117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:33.144672 8113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:33.145108 8105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:33.145340 8105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:33.146466 8105 hybrid_clock.cc:648] HybridClock initialized: now 1786301553146455 us; error 27 us; skew 500 ppm
I20260809 18:52:33.147632 8105 webserver.cc:460] Webserver started at http://127.23.218.195:35273/ using document root <none> and password file <none>
I20260809 18:52:33.147845 8105 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:33.147889 8105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:33.148005 8105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:33.149031 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "fe70504cd0db496b807b5222a6609eaf"
format_stamp: "Formatted at 2026-08-09 18:52:33 on dist-test-slave-6zbq"
server_key: "4cb98bb6bb2eec18f9dc4798513a25ce"
server_key_iv: "4d1770177df8294050ddf2fc827eabed"
server_key_version: "encryptionkey@0"
I20260809 18:52:33.149356 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "fe70504cd0db496b807b5222a6609eaf"
format_stamp: "Formatted at 2026-08-09 18:52:33 on dist-test-slave-6zbq"
server_key: "4cb98bb6bb2eec18f9dc4798513a25ce"
server_key_iv: "4d1770177df8294050ddf2fc827eabed"
server_key_version: "encryptionkey@0"
I20260809 18:52:33.150575 8105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:52:33.151202 8124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:33.151337 8105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:33.151414 8105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "fe70504cd0db496b807b5222a6609eaf"
format_stamp: "Formatted at 2026-08-09 18:52:33 on dist-test-slave-6zbq"
server_key: "4cb98bb6bb2eec18f9dc4798513a25ce"
server_key_iv: "4d1770177df8294050ddf2fc827eabed"
server_key_version: "encryptionkey@0"
I20260809 18:52:33.151479 8105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:33.154556 8105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:33.154768 8105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:33.154874 8105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:33.155042 8105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:33.155314 8105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:33.155344 8105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:33.155373 8105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:33.155396 8105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:33.160714 8105 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:38439
I20260809 18:52:33.160772 8277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:38439 every 8 connection(s)
I20260809 18:52:33.161017 8105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:52:33.163658 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:33.161652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:39207 (local address 127.23.218.254:35555)
0809 18:52:33.161699 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:33.161701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:33.161892 (+ 191us) server_negotiation.cc:407] Connection header received
0809 18:52:33.162365 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:33.162366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:33.162374 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:33.162423 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:33.163426 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:33.163428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:33.163428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:52:33.163432 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:33.163448 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:52:33.163453 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:33.163504 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:33.163567 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 18:52:33.163581 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 18:52:33.163811 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:33.161618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:35555 (local address 127.23.218.195:39207)
0809 18:52:33.161794 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:33.161822 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:33.162246 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:33.162487 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:33.162491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:33.162501 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:33.163292 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:52:33.163292 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:33.163318 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:52:33.163342 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:33.163505 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:33.163509 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:33.163512 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:52:33.163577 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 18:52:33.163634 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260809 18:52:33.164214 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 8105
I20260809 18:52:33.164265 8279 heartbeater.cc:347] Connected to a master server at 127.23.218.254:35555
I20260809 18:52:33.164314 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:52:33.164345 8279 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:33.164474 8279 heartbeater.cc:511] Master 127.23.218.254:35555 requested a full tablet report, sending...
I20260809 18:52:33.164479 24427 external_mini_cluster.cc:1428] Setting key 6693a19c9104c632d3f66db27b100fe4
I20260809 18:52:33.164809 7619 ts_manager.cc:194] Registered new tserver with Master: fe70504cd0db496b807b5222a6609eaf (127.23.218.195:38439)
I20260809 18:52:33.165272 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 18:52:33.166925 7687 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:52:33.166523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.23.218.254:35555)
0809 18:52:33.166560 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:33.166562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:33.166658 (+ 96us) server_negotiation.cc:407] Connection header received
0809 18:52:33.166717 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:33.166718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:33.166723 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:33.166766 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:33.166872 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47514: BlockingRecv error: recv got EOF from 127.0.0.1:47514 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:52:33.166998 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 7720
I20260809 18:52:33.181932 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 7909
I20260809 18:52:33.192445 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 8105
I20260809 18:52:33.198957 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 7579
2026-08-09T18:52:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (524 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:52:33 dist-test-slave-6zbq krb5kdc[8305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:52:33 dist-test-slave-6zbq krb5kdc[8305](info): set up 2 sockets
Aug 09 18:52:33 dist-test-slave-6zbq krb5kdc[8305](info): commencing operation
krb5kdc: starting...
W20260809 18:52:34.318871 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:52:34 dist-test-slave-6zbq krb5kdc[8305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:52:34.424805 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:43077
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:43077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:34.503216 8324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:34.503393 8324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:34.503414 8324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:34.504454 8324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:52:34.504487 8324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:34.504499 8324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:34.504510 8324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:52:34.504523 8324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:52:34.505908 8324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:43077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:43077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:34.506081 8324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:34.506235 8324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:34.510643 8338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:34.510690 8335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:34.510679 8334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:34.510722 8324 server_base.cc:1034] running on GCE node
I20260809 18:52:34.511020 8324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:34.511272 8324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:34.512409 8324 hybrid_clock.cc:648] HybridClock initialized: now 1786301554512393 us; error 28 us; skew 500 ppm
Aug 09 18:52:34 dist-test-slave-6zbq krb5kdc[8305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301554, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:34.514600 8324 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:52:34.515002 8324 webserver.cc:460] Webserver started at http://127.23.218.254:39337/ using document root <none> and password file <none>
I20260809 18:52:34.515195 8324 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:34.515240 8324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:34.515362 8324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:34.516292 8324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "dbb307a636ab49bb9990fd545416a541"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "c1a574d453108526231f9283237814e5"
server_key_iv: "ca8c02c9ca0d811eccec5b87428b7200"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.516603 8324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "dbb307a636ab49bb9990fd545416a541"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "c1a574d453108526231f9283237814e5"
server_key_iv: "ca8c02c9ca0d811eccec5b87428b7200"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.517778 8324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:52:34.518412 8348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:34.518553 8324 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:34.518635 8324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "dbb307a636ab49bb9990fd545416a541"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "c1a574d453108526231f9283237814e5"
server_key_iv: "ca8c02c9ca0d811eccec5b87428b7200"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.518707 8324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:34.532382 8324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:34.533991 8324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:34.534106 8324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:34.537737 8324 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:43077
I20260809 18:52:34.537774 8433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:43077 every 8 connection(s)
I20260809 18:52:34.538049 8324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:52:34.538494 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 8324
I20260809 18:52:34.538587 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:52:34.538750 24427 external_mini_cluster.cc:1428] Setting key eb8f5efe793aaf0c0935b8a909523ecf
I20260809 18:52:34.539040 8434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:34.541947 8434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541: Bootstrap starting.
I20260809 18:52:34.542627 8434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:34.542888 8434 log.cc:824] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:34.543546 8434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541: No bootstrap required, opened a new log
Aug 09 18:52:34 dist-test-slave-6zbq krb5kdc[8305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:34.545555 8434 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb307a636ab49bb9990fd545416a541" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43077 } }
I20260809 18:52:34.545730 8434 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:34.545783 8434 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbb307a636ab49bb9990fd545416a541, State: Initialized, Role: FOLLOWER
I20260809 18:52:34.545910 8434 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb307a636ab49bb9990fd545416a541" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43077 } }
I20260809 18:52:34.545985 8434 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:52:34.546008 8434 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:52:34.546054 8434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:34.546643 8434 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb307a636ab49bb9990fd545416a541" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43077 } }
I20260809 18:52:34.546730 8434 leader_election.cc:302] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbb307a636ab49bb9990fd545416a541; no voters:
I20260809 18:52:34.546873 8434 leader_election.cc:288] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:52:34.546929 8440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:34.547086 8434 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:52:34.547114 8440 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [term 1 LEADER]: Becoming Leader. State: Replica: dbb307a636ab49bb9990fd545416a541, State: Running, Role: LEADER
I20260809 18:52:34.547297 8440 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb307a636ab49bb9990fd545416a541" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43077 } }
I20260809 18:52:34.547632 8440 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbb307a636ab49bb9990fd545416a541. Latest consensus state: current_term: 1 leader_uuid: "dbb307a636ab49bb9990fd545416a541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb307a636ab49bb9990fd545416a541" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43077 } } }
I20260809 18:52:34.547641 8443 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbb307a636ab49bb9990fd545416a541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbb307a636ab49bb9990fd545416a541" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43077 } } }
I20260809 18:52:34.547812 8440 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:34.547825 8443 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541 [sys.catalog]: This master's current role is: LEADER
I20260809 18:52:34.548017 8452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:52:34.548489 8452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:52:34.548780 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:34.539204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.23.218.254:43077)
0809 18:52:34.539356 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:34.539364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:34.539388 (+ 24us) server_negotiation.cc:407] Connection header received
0809 18:52:34.539758 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:34.539764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:34.539805 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:34.539904 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:34.540627 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.541468 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.542485 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.542892 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.542987 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:34.545379 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:34.545383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:34.545383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:34.545393 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:34.547079 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:34.547468 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:34.547471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:34.547471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:34.547570 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:34.547772 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:34.547773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:34.547773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:34.548062 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:34.548204 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:34.548453 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 18:52:34.548598 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260809 18:52:34.550202 8452 catalog_manager.cc:1347] Generated new cluster ID: bf8c74a2bf624921b789acb81ce82f6c
I20260809 18:52:34.550252 8452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:52:34.553754 8452 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:52:34.554448 8452 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:52:34.564018 8452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541: Generated new TSK 0
I20260809 18:52:34.564165 8452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:52:34.589807 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43077
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:34.670431 8473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:34.670606 8473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:34.670627 8473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:34.671743 8473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:34.671782 8473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:34.671805 8473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:52:34.673173 8473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:34.673382 8473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:34.673569 8473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:34.678068 8479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:34.678081 8480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:34.678252 8473 server_base.cc:1034] running on GCE node
W20260809 18:52:34.678259 8482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:34.678488 8473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:34.678740 8473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:34.679876 8473 hybrid_clock.cc:648] HybridClock initialized: now 1786301554679861 us; error 30 us; skew 500 ppm
Aug 09 18:52:34 dist-test-slave-6zbq krb5kdc[8305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301554, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:34.682236 8473 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:52:34.682636 8473 webserver.cc:460] Webserver started at http://127.23.218.193:39691/ using document root <none> and password file <none>
I20260809 18:52:34.682803 8473 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:34.682839 8473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:34.682921 8473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:34.683846 8473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "155603c0e6914a2a95134e385de59de0"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "a9acf0a6b50242de11392013e81d03f7"
server_key_iv: "196443ec2ef34bc2ee2008a14e661bb0"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.684123 8473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "155603c0e6914a2a95134e385de59de0"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "a9acf0a6b50242de11392013e81d03f7"
server_key_iv: "196443ec2ef34bc2ee2008a14e661bb0"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.685220 8473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:52:34.685832 8492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:34.685974 8473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:34.686057 8473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "155603c0e6914a2a95134e385de59de0"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "a9acf0a6b50242de11392013e81d03f7"
server_key_iv: "196443ec2ef34bc2ee2008a14e661bb0"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.686116 8473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:34.696285 8473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:34.698184 8473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:34.698309 8473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:34.698514 8473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:34.698782 8473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:34.698814 8473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:34.698839 8473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:34.698856 8473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:34.704066 8473 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:39853
I20260809 18:52:34.704121 8661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:39853 every 8 connection(s)
I20260809 18:52:34.704381 8473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
Aug 09 18:52:34 dist-test-slave-6zbq krb5kdc[8305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301554, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:34.712296 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:34.704980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:37219 (local address 127.23.218.254:43077)
0809 18:52:34.705020 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:34.705022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:34.705192 (+ 170us) server_negotiation.cc:407] Connection header received
0809 18:52:34.705633 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:34.705634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:34.705646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:34.705696 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:34.706577 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.707061 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.707997 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.708202 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.708255 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:34.710042 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:34.710043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:34.710043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:34.710051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:34.711108 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:34.711451 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:34.711452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:34.711452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:34.711500 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:34.711716 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:34.711716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:34.711717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:34.711818 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:34.711873 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:34.712114 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 18:52:34.712206 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:52:34.712411 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:34.704951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:43077 (local address 127.23.218.193:37219)
0809 18:52:34.705085 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:34.705118 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:34.705545 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:34.705759 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:34.705763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:34.706273 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:34.706488 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:34.706493 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.707143 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.707144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:34.707913 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:34.707915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.708272 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.708273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:34.708394 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:34.708750 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:34.708758 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:34.709953 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:34.711181 (+ 1228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:34.711188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:34.711190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:34.711388 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:34.711553 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:34.711553 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:34.711554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:34.711660 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:34.711873 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:34.711879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:34.712002 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 18:52:34.712086 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 18:52:34.712224 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260809 18:52:34.712831 8662 heartbeater.cc:347] Connected to a master server at 127.23.218.254:43077
I20260809 18:52:34.712911 8662 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:34.713033 8662 heartbeater.cc:511] Master 127.23.218.254:43077 requested a full tablet report, sending...
I20260809 18:52:34.713495 8371 ts_manager.cc:194] Registered new tserver with Master: 155603c0e6914a2a95134e385de59de0 (127.23.218.193:39853)
I20260809 18:52:34.714007 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 8473
I20260809 18:52:34.714082 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:52:34.714092 8371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:37219
I20260809 18:52:34.714253 24427 external_mini_cluster.cc:1428] Setting key 8386da8c9f2868f43b130a39c23729dd
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:52:34.736286 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43077
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:34.813174 8667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:34.813342 8667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:34.813364 8667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:34.814463 8667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:34.814502 8667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:34.814527 8667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:52:34.815907 8667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:34.816102 8667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:34.816289 8667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:34.820729 8676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:34.820739 8674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:34.820735 8673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:34.820896 8667 server_base.cc:1034] running on GCE node
I20260809 18:52:34.821045 8667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:34.821272 8667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:34.822383 8667 hybrid_clock.cc:648] HybridClock initialized: now 1786301554822364 us; error 34 us; skew 500 ppm
Aug 09 18:52:34 dist-test-slave-6zbq krb5kdc[8305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301554, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:34.824591 8667 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:52:34.824959 8667 webserver.cc:460] Webserver started at http://127.23.218.194:33127/ using document root <none> and password file <none>
I20260809 18:52:34.825165 8667 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:34.825207 8667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:34.825301 8667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:34.826315 8667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "d637dbfc1e7e4e12abe2badbea9c760b"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "d9dcc219fd85b9ed6a9ecd0f226e35b0"
server_key_iv: "4749fc59bef06b704b5c91f3cd593a18"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.826653 8667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "d637dbfc1e7e4e12abe2badbea9c760b"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "d9dcc219fd85b9ed6a9ecd0f226e35b0"
server_key_iv: "4749fc59bef06b704b5c91f3cd593a18"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.827770 8667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:52:34.828377 8685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:34.828508 8667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:52:34.828572 8667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "d637dbfc1e7e4e12abe2badbea9c760b"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "d9dcc219fd85b9ed6a9ecd0f226e35b0"
server_key_iv: "4749fc59bef06b704b5c91f3cd593a18"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.828634 8667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:34.842979 8667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:34.844810 8667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:34.844925 8667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:34.845114 8667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:34.845367 8667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:34.845399 8667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:34.845417 8667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:34.845439 8667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:34.850872 8667 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:45727
I20260809 18:52:34.850935 8859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:45727 every 8 connection(s)
I20260809 18:52:34.851174 8667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
Aug 09 18:52:34 dist-test-slave-6zbq krb5kdc[8305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301554, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:34.858807 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:34.851875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:42589 (local address 127.23.218.254:43077)
0809 18:52:34.851930 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:34.851932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:34.852101 (+ 169us) server_negotiation.cc:407] Connection header received
0809 18:52:34.852513 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:34.852514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:34.852523 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:34.852580 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:34.853360 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.853848 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.854735 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.854918 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.854965 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:34.856750 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:34.856752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:34.856752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:34.856758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:34.857788 (+ 1030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:34.858098 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:34.858098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:34.858099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:34.858148 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:34.858313 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:34.858313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:34.858313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:34.858447 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:34.858497 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:34.858688 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 18:52:34.858746 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:52:34.858932 8865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:34.851840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:43077 (local address 127.23.218.194:42589)
0809 18:52:34.851994 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:34.852026 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:34.852442 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:34.852620 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:34.852623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:34.853117 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:34.853290 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:34.853294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.853909 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.853910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:34.854670 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:34.854672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:34.854974 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:34.854974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:34.855065 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:34.855469 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:34.855477 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:34.856670 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:34.857849 (+ 1179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:34.857854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:34.857855 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:34.858042 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:34.858193 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:34.858193 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:34.858194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:34.858272 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:34.858493 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:34.858497 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:34.858592 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 18:52:34.858669 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 18:52:34.858774 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260809 18:52:34.859360 8860 heartbeater.cc:347] Connected to a master server at 127.23.218.254:43077
I20260809 18:52:34.859453 8860 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:34.859622 8860 heartbeater.cc:511] Master 127.23.218.254:43077 requested a full tablet report, sending...
I20260809 18:52:34.859963 8371 ts_manager.cc:194] Registered new tserver with Master: d637dbfc1e7e4e12abe2badbea9c760b (127.23.218.194:45727)
I20260809 18:52:34.860313 8371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:42589
I20260809 18:52:34.860483 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 8667
I20260809 18:52:34.860574 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:52:34.860730 24427 external_mini_cluster.cc:1428] Setting key f3f6e833d7af93c740b4e72508441f9a
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:52:34.882942 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43077
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:52:34.960646 8870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:52:34.960824 8870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:52:34.960846 8870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:52:34.961907 8870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:52:34.961939 8870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:52:34.961963 8870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:52:34.963346 8870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:52:34.963519 8870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:52:34.963678 8870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:52:34.968207 8875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:34.968179 8876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:52:34.968227 8879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:52:34.968284 8870 server_base.cc:1034] running on GCE node
I20260809 18:52:34.968564 8870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:52:34.968768 8870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:52:34.969898 8870 hybrid_clock.cc:648] HybridClock initialized: now 1786301554969890 us; error 25 us; skew 500 ppm
Aug 09 18:52:34 dist-test-slave-6zbq krb5kdc[8305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301554, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:52:34.972244 8870 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:52:34.972592 8870 webserver.cc:460] Webserver started at http://127.23.218.195:38519/ using document root <none> and password file <none>
I20260809 18:52:34.972770 8870 fs_manager.cc:359] Metadata directory not provided
I20260809 18:52:34.972808 8870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:52:34.972891 8870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:52:34.973778 8870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "12be942ee65c47eda3e1fe256dff971f"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "f630574da177935a9feaa82b08bb9544"
server_key_iv: "5c69671b976339372544284bc238867e"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.974053 8870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "12be942ee65c47eda3e1fe256dff971f"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "f630574da177935a9feaa82b08bb9544"
server_key_iv: "5c69671b976339372544284bc238867e"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.975219 8870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:52:34.975823 8890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:34.975963 8870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:52:34.976044 8870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "12be942ee65c47eda3e1fe256dff971f"
format_stamp: "Formatted at 2026-08-09 18:52:34 on dist-test-slave-6zbq"
server_key: "f630574da177935a9feaa82b08bb9544"
server_key_iv: "5c69671b976339372544284bc238867e"
server_key_version: "encryptionkey@0"
I20260809 18:52:34.976102 8870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:52:34.998450 8870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:52:35.000385 8870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:52:35.000504 8870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:52:35.000684 8870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:52:35.000945 8870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:52:35.000977 8870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:35.001004 8870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:52:35.001022 8870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:52:35.006110 8870 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:38439
I20260809 18:52:35.006171 9069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:38439 every 8 connection(s)
I20260809 18:52:35.006464 8870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:52:35.006915 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 8870
I20260809 18:52:35.007014 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:52:35.007208 24427 external_mini_cluster.cc:1428] Setting key dc1a7d678b5db970b5c082012291bf6e
Aug 09 18:52:35 dist-test-slave-6zbq krb5kdc[8305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301554, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:52:35.014858 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.007375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:50279 (local address 127.23.218.254:43077)
0809 18:52:35.007420 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:35.007422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:35.007432 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:52:35.007939 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:35.007940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:35.007952 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:35.008007 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:35.008875 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.009357 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.010276 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.010550 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.010624 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.012545 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:35.012547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:35.012547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:35.012554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:35.013615 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:35.013962 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:35.013963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:35.013963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:35.014017 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:35.014245 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:35.014245 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:35.014246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:35.014386 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:35.014449 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:35.014693 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 18:52:35.014774 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:52:35.014997 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.007087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:43077 (local address 127.23.218.195:50279)
0809 18:52:35.007260 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:35.007305 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:35.007845 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:35.008075 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:35.008079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:35.008617 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:52:35.008803 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.008808 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.009418 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.009419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.010182 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.010184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.010630 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.010631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.010742 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.011165 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:52:35.011173 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:52:35.012449 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:52:35.013687 (+ 1238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:35.013694 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:35.013696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:35.013897 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:35.014071 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:52:35.014071 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:52:35.014072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:52:35.014185 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:52:35.014439 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:52:35.014445 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:52:35.014573 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 18:52:35.014659 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 18:52:35.014803 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260809 18:52:35.015407 9070 heartbeater.cc:347] Connected to a master server at 127.23.218.254:43077
I20260809 18:52:35.015487 9070 heartbeater.cc:464] Registering TS with master...
I20260809 18:52:35.015645 9070 heartbeater.cc:511] Master 127.23.218.254:43077 requested a full tablet report, sending...
I20260809 18:52:35.015967 8371 ts_manager.cc:194] Registered new tserver with Master: 12be942ee65c47eda3e1fe256dff971f (127.23.218.195:38439)
I20260809 18:52:35.016332 8371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:50279
I20260809 18:52:35.018798 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:52:35.024998 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.019945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54250 (local address 127.23.218.254:43077)
0809 18:52:35.020007 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:35.020009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:35.020098 (+ 89us) server_negotiation.cc:407] Connection header received
0809 18:52:35.020165 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:35.020166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:35.020175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:35.020220 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:52:35.020755 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.021242 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.021933 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.022111 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.022163 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.023101 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:52:35.023102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:52:35.023102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:52:35.023108 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:52:35.024261 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:35.024493 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:35.024493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:35.024494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:35.024534 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:52:35.024636 (+ 102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:52:35.024636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:52:35.024636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:52:35.024745 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:52:35.024812 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:35.024881 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 18:52:35.024933 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 18:52:35.026213 8371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:52:35.027711 8371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:52:35.033308 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.029583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:45727 (local address 127.0.0.1:57206)
0809 18:52:35.029708 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:35.029739 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:35.029799 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:35.030228 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:35.030232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:35.030234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:35.030294 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.030295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.031372 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.031373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.032348 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.032352 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.033028 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.033029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.033109 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.033122 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 18:52:35.033181 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 18:52:35.033232 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260809 18:52:35.033320 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.029728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.23.218.194:45727)
0809 18:52:35.030073 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:35.030077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:35.030086 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:35.030117 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:35.030119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:35.030132 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:35.030209 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:35.030436 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.031293 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.032434 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.032950 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.033016 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.033070 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:35.033203 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 18:52:35.033246 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":79,"threads_started":1}
I20260809 18:52:35.034152 9085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.029782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:38439 (local address 127.0.0.1:47608)
0809 18:52:35.029872 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:35.029881 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:35.030048 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:35.030546 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:35.030547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:35.030549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:35.030599 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.030601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.031882 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.031883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.033037 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.033040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.033906 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.033907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.033970 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.033974 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:35.034022 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 18:52:35.034059 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":44,"threads_started":1}
I20260809 18:52:35.034296 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.029897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47608 (local address 127.23.218.195:38439)
0809 18:52:35.030251 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:35.030254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:35.030262 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:35.030312 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:35.030314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:35.030327 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:35.030667 (+ 340us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:35.030731 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.031768 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.033133 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.033821 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.033885 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.033930 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:35.034165 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 18:52:35.034208 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":73,"threads_started":1}
I20260809 18:52:35.034308 8762 tablet_service.cc:1467] Processing CreateTablet for tablet 8210c26e64c947eb89cb668fc9d55ade (DEFAULT_TABLE table=test-table [id=de828cf714f748ecb3dc5f79541dfacb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:35.034590 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.029905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39853 (local address 127.0.0.1:57816)
0809 18:52:35.030421 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:35.030430 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:35.030484 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:35.030839 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:35.030840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:35.030842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:35.030887 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.030888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.032350 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.032351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.033406 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.033409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.034358 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.034360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.034432 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.034437 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:52:35.034486 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 18:52:35.034527 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":61,"threads_started":1}
I20260809 18:52:35.034736 8762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8210c26e64c947eb89cb668fc9d55ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:35.034950 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.030028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.23.218.193:39853)
0809 18:52:35.030190 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:35.030194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:35.030546 (+ 352us) server_negotiation.cc:407] Connection header received
0809 18:52:35.030619 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:35.030622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:35.030638 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:35.030835 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:35.031004 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.032094 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.033492 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.034189 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.034252 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.034301 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:35.034736 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 18:52:35.034892 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260809 18:52:35.034984 8972 tablet_service.cc:1467] Processing CreateTablet for tablet 8210c26e64c947eb89cb668fc9d55ade (DEFAULT_TABLE table=test-table [id=de828cf714f748ecb3dc5f79541dfacb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:35.035296 8972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8210c26e64c947eb89cb668fc9d55ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:35.035473 8560 tablet_service.cc:1467] Processing CreateTablet for tablet 8210c26e64c947eb89cb668fc9d55ade (DEFAULT_TABLE table=test-table [id=de828cf714f748ecb3dc5f79541dfacb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:52:35.035719 8560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8210c26e64c947eb89cb668fc9d55ade. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:52:35.037948 9093 tablet_bootstrap.cc:492] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0: Bootstrap starting.
I20260809 18:52:35.038427 9095 tablet_bootstrap.cc:492] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: Bootstrap starting.
I20260809 18:52:35.038463 9096 tablet_bootstrap.cc:492] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: Bootstrap starting.
I20260809 18:52:35.038619 9093 tablet_bootstrap.cc:654] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:35.038930 9096 tablet_bootstrap.cc:654] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:35.039134 9095 tablet_bootstrap.cc:654] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: Neither blocks nor log segments found. Creating new log.
I20260809 18:52:35.039201 9093 log.cc:824] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:35.039435 9096 log.cc:824] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:35.039709 9095 log.cc:824] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: Log is configured to *not* fsync() on all Append() calls
I20260809 18:52:35.039994 9093 tablet_bootstrap.cc:492] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0: No bootstrap required, opened a new log
I20260809 18:52:35.040051 9093 ts_tablet_manager.cc:1397] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:52:35.040315 9095 tablet_bootstrap.cc:492] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: No bootstrap required, opened a new log
I20260809 18:52:35.040367 9095 ts_tablet_manager.cc:1397] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:52:35.041539 9093 raft_consensus.cc:348] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } }
I20260809 18:52:35.041674 9093 raft_consensus.cc:374] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:35.041718 9093 raft_consensus.cc:729] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 155603c0e6914a2a95134e385de59de0, State: Initialized, Role: FOLLOWER
I20260809 18:52:35.041838 9093 consensus_queue.cc:260] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } }
I20260809 18:52:35.041836 9095 raft_consensus.cc:348] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } }
I20260809 18:52:35.041960 9095 raft_consensus.cc:374] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:35.042018 9095 raft_consensus.cc:729] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d637dbfc1e7e4e12abe2badbea9c760b, State: Initialized, Role: FOLLOWER
I20260809 18:52:35.042104 8662 heartbeater.cc:503] Master 127.23.218.254:43077 was elected leader, sending a full tablet report...
I20260809 18:52:35.042102 9093 ts_tablet_manager.cc:1428] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:52:35.042182 9095 consensus_queue.cc:260] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } }
I20260809 18:52:35.042460 8860 heartbeater.cc:503] Master 127.23.218.254:43077 was elected leader, sending a full tablet report...
I20260809 18:52:35.042729 9096 tablet_bootstrap.cc:492] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: No bootstrap required, opened a new log
I20260809 18:52:35.042771 9096 ts_tablet_manager.cc:1397] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 18:52:35.042970 9095 ts_tablet_manager.cc:1428] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 18:52:35.044234 9096 raft_consensus.cc:348] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } }
I20260809 18:52:35.044392 9096 raft_consensus.cc:374] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:52:35.044471 9096 raft_consensus.cc:729] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12be942ee65c47eda3e1fe256dff971f, State: Initialized, Role: FOLLOWER
I20260809 18:52:35.044587 9096 consensus_queue.cc:260] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } }
I20260809 18:52:35.044799 9096 ts_tablet_manager.cc:1428] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:52:35.044836 9070 heartbeater.cc:503] Master 127.23.218.254:43077 was elected leader, sending a full tablet report...
W20260809 18:52:35.101653 8861 tablet.cc:2369] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:52:35.205013 8663 tablet.cc:2369] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:52:35.257620 9071 tablet.cc:2369] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:52:35.286707 9101 raft_consensus.cc:482] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:52:35.286796 9101 raft_consensus.cc:504] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } }
I20260809 18:52:35.287093 9101 leader_election.cc:288] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12be942ee65c47eda3e1fe256dff971f (127.23.218.195:38439), 155603c0e6914a2a95134e385de59de0 (127.23.218.193:39853)
I20260809 18:52:35.289973 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.287414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:38109 (local address 127.23.218.193:39853)
0809 18:52:35.287474 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:35.287476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:35.287593 (+ 117us) server_negotiation.cc:407] Connection header received
0809 18:52:35.287646 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:35.287647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:35.287661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:35.287709 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:35.287843 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.288282 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.289239 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.289677 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.289730 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.289738 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:35.289874 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 18:52:35.289909 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 18:52:35.289966 9103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.287372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39853 (local address 127.23.218.194:38109)
0809 18:52:35.287534 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:35.287544 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:35.287591 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:35.287717 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:35.287718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:35.287720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:35.287788 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.287789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.288352 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.288353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.289162 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.289164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.289744 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.289745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.289813 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.289817 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:35.289857 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 18:52:35.289886 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260809 18:52:35.290014 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.287323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:47691 (local address 127.23.218.195:38439)
0809 18:52:35.287384 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:35.287386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:35.287395 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:52:35.287422 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:35.287423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:35.287433 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:35.287491 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:52:35.287652 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.288221 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.289266 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.289697 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.289766 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.289775 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:35.289924 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 18:52:35.289958 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 18:52:35.290040 8865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.287249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:38439 (local address 127.23.218.194:47691)
0809 18:52:35.287298 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:35.287309 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:52:35.287348 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:52:35.287520 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:52:35.287521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:52:35.287523 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:52:35.287586 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.287588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.288290 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.288290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.289176 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:52:35.289179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.289773 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.289774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:52:35.289851 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.289855 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:52:35.289906 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 18:52:35.289944 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":3}
I20260809 18:52:35.290364 9001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8210c26e64c947eb89cb668fc9d55ade" candidate_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12be942ee65c47eda3e1fe256dff971f" is_pre_election: true
I20260809 18:52:35.290433 8588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8210c26e64c947eb89cb668fc9d55ade" candidate_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "155603c0e6914a2a95134e385de59de0" is_pre_election: true
I20260809 18:52:35.290531 9001 raft_consensus.cc:2454] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d637dbfc1e7e4e12abe2badbea9c760b in term 0.
I20260809 18:52:35.290557 8588 raft_consensus.cc:2454] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d637dbfc1e7e4e12abe2badbea9c760b in term 0.
I20260809 18:52:35.290768 8688 leader_election.cc:302] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 155603c0e6914a2a95134e385de59de0, d637dbfc1e7e4e12abe2badbea9c760b; no voters:
I20260809 18:52:35.290876 9101 raft_consensus.cc:2781] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:52:35.290943 9101 raft_consensus.cc:482] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:52:35.290966 9101 raft_consensus.cc:3037] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:35.291635 9101 raft_consensus.cc:504] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } }
I20260809 18:52:35.291750 9101 leader_election.cc:288] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [CANDIDATE]: Term 1 election: Requested vote from peers 12be942ee65c47eda3e1fe256dff971f (127.23.218.195:38439), 155603c0e6914a2a95134e385de59de0 (127.23.218.193:39853)
I20260809 18:52:35.291911 9001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8210c26e64c947eb89cb668fc9d55ade" candidate_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12be942ee65c47eda3e1fe256dff971f"
I20260809 18:52:35.291936 8588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8210c26e64c947eb89cb668fc9d55ade" candidate_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "155603c0e6914a2a95134e385de59de0"
I20260809 18:52:35.291981 9001 raft_consensus.cc:3037] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:35.292003 8588 raft_consensus.cc:3037] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:52:35.292614 9001 raft_consensus.cc:2454] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d637dbfc1e7e4e12abe2badbea9c760b in term 1.
I20260809 18:52:35.292702 8588 raft_consensus.cc:2454] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d637dbfc1e7e4e12abe2badbea9c760b in term 1.
I20260809 18:52:35.292784 8688 leader_election.cc:302] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12be942ee65c47eda3e1fe256dff971f, d637dbfc1e7e4e12abe2badbea9c760b; no voters:
I20260809 18:52:35.292850 9101 raft_consensus.cc:2781] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:52:35.292968 9101 raft_consensus.cc:686] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 1 LEADER]: Becoming Leader. State: Replica: d637dbfc1e7e4e12abe2badbea9c760b, State: Running, Role: LEADER
I20260809 18:52:35.293051 9101 consensus_queue.cc:237] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } }
I20260809 18:52:35.293841 8368 catalog_manager.cc:5495] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b reported cstate change: term changed from 0 to 1, leader changed from <none> to d637dbfc1e7e4e12abe2badbea9c760b (127.23.218.194). New cstate: current_term: 1 leader_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d637dbfc1e7e4e12abe2badbea9c760b" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 } health_report { overall_health: UNKNOWN } } }
I20260809 18:52:35.315677 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.313552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.23.218.194:45727)
0809 18:52:35.313604 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:52:35.313606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:52:35.313614 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:52:35.313688 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:52:35.313689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:52:35.313705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:52:35.313759 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:52:35.313878 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.314295 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.315016 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:52:35.315194 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:52:35.315245 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:52:35.315388 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:52:35.315472 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:52:35.315539 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 18:52:35.315577 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 18:52:35.315594 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:52:35.317270 9001 raft_consensus.cc:1261] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [term 1 FOLLOWER]: Refusing update from remote peer d637dbfc1e7e4e12abe2badbea9c760b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:35.317270 8588 raft_consensus.cc:1261] T 8210c26e64c947eb89cb668fc9d55ade P 155603c0e6914a2a95134e385de59de0 [term 1 FOLLOWER]: Refusing update from remote peer d637dbfc1e7e4e12abe2badbea9c760b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:52:35.317571 9101 consensus_queue.cc:1035] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [LEADER]: Connected to new peer: Peer: permanent_uuid: "155603c0e6914a2a95134e385de59de0" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:35.317699 9104 consensus_queue.cc:1035] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [LEADER]: Connected to new peer: Peer: permanent_uuid: "12be942ee65c47eda3e1fe256dff971f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:52:35.319952 9111 mvcc.cc:205] Tried to move back new op lower bound from 7316691170577203200 to 7316691170480795648. Current Snapshot: MvccSnapshot[applied={T|T < 7316691170577203200}]
I20260809 18:52:35.320698 9113 mvcc.cc:205] Tried to move back new op lower bound from 7316691170577203200 to 7316691170480795648. Current Snapshot: MvccSnapshot[applied={T|T < 7316691170577203200}]
I20260809 18:52:35.322320 8371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:35.322429 8371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:52:35.323043 8371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbb307a636ab49bb9990fd545416a541: Sending DeleteTablet for 3 replicas of tablet 8210c26e64c947eb89cb668fc9d55ade
I20260809 18:52:35.323376 8972 tablet_service.cc:1514] Processing DeleteTablet for tablet 8210c26e64c947eb89cb668fc9d55ade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:35 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:47608
I20260809 18:52:35.323418 8560 tablet_service.cc:1514] Processing DeleteTablet for tablet 8210c26e64c947eb89cb668fc9d55ade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:35 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:57816
I20260809 18:52:35.323433 8762 tablet_service.cc:1514] Processing DeleteTablet for tablet 8210c26e64c947eb89cb668fc9d55ade with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:52:35 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:57206
I20260809 18:52:35.323606 9127 tablet_replica.cc:330] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: stopping tablet replica
I20260809 18:52:35.323606 9125 tablet_replica.cc:330] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: stopping tablet replica
I20260809 18:52:35.323690 9127 raft_consensus.cc:2229] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:52:35.323706 9125 raft_consensus.cc:2229] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:52:35.323802 9125 raft_consensus.cc:2258] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:35.323844 9127 raft_consensus.cc:2258] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:52:35.323921 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 8473
I20260809 18:52:35.324553 9125 ts_tablet_manager.cc:1905] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:35.326231 9125 ts_tablet_manager.cc:1918] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:35.326319 9125 log.cc:1196] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/8210c26e64c947eb89cb668fc9d55ade
I20260809 18:52:35.326674 9125 ts_tablet_manager.cc:1939] T 8210c26e64c947eb89cb668fc9d55ade P 12be942ee65c47eda3e1fe256dff971f: Deleting consensus metadata
I20260809 18:52:35.327037 8350 catalog_manager.cc:4841] TS 12be942ee65c47eda3e1fe256dff971f (127.23.218.195:38439): tablet 8210c26e64c947eb89cb668fc9d55ade (table test-table [id=de828cf714f748ecb3dc5f79541dfacb]) successfully deleted
I20260809 18:52:35.330430 9127 ts_tablet_manager.cc:1905] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:52:35.332070 9127 ts_tablet_manager.cc:1918] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:52:35.332152 9127 log.cc:1196] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/8210c26e64c947eb89cb668fc9d55ade
I20260809 18:52:35.332428 9127 ts_tablet_manager.cc:1939] T 8210c26e64c947eb89cb668fc9d55ade P d637dbfc1e7e4e12abe2badbea9c760b: Deleting consensus metadata
I20260809 18:52:35.332795 8350 catalog_manager.cc:4841] TS d637dbfc1e7e4e12abe2badbea9c760b (127.23.218.194:45727): tablet 8210c26e64c947eb89cb668fc9d55ade (table test-table [id=de828cf714f748ecb3dc5f79541dfacb]) successfully deleted
W20260809 18:52:35.335314 8350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:39853 (error 108)
I20260809 18:52:35.335554 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 8667
I20260809 18:52:35.338937 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:52:35.335664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39853 (local address 127.0.0.1:57828)
0809 18:52:35.338872 (+ 3208us) negotiation.cc:107] Waiting for socket to connect
0809 18:52:35.338911 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:39853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3190}
W20260809 18:52:35.339044 8350 catalog_manager.cc:4584] TS 155603c0e6914a2a95134e385de59de0 (127.23.218.193:39853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8210c26e64c947eb89cb668fc9d55ade: Network error: Client connection negotiation failed: client connection to 127.23.218.193:39853: connect: Connection refused (error 111)
I20260809 18:52:35.347183 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 8870
I20260809 18:52:35.357048 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 8324
2026-08-09T18:52:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2160 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:52:35 dist-test-slave-6zbq krb5kdc[9134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:52:35 dist-test-slave-6zbq krb5kdc[9134](info): set up 2 sockets
Aug 09 18:52:35 dist-test-slave-6zbq krb5kdc[9134](info): commencing operation
krb5kdc: starting...
W20260809 18:52:36.493889 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:52:36 dist-test-slave-6zbq krb5kdc[9134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:52:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/rangeradmin_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/rangeradmin_127.23.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/rangerlookup_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/rangerlookup_127.23.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/HTTP_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/HTTP_127.23.218.212@KRBTEST.COM.keytab.
I20260809 18:52:36.640873 24427 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskUXKsiJ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/postgres -l logfile start
2026-08-09 18:52:38.777 UTC [9186] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 18:52:38.777 UTC [9186] LOG: listening on IPv4 address "127.23.218.212", port 46795
2026-08-09 18:52:38.780 UTC [9186] LOG: listening on Unix socket "/tmp/.s.PGSQL.46795"
2026-08-09 18:52:38.799 UTC [9189] LOG: database system was shut down at 2026-08-09 18:52:37 UTC
2026-08-09 18:52:38.802 UTC [9186] LOG: database system is ready to accept connections
I20260809 18:52:39.860733 24427 mini_postgres.cc:96] Postgres bound to 46795
2026-08-09 18:52:39.868 UTC [9199] FATAL: database "slave" does not exist
127.23.218.212:46795 - accepting connections
I20260809 18:52:39.869555 24427 mini_ranger.cc:161] Starting Ranger...
I20260809 18:52:39.883936 24427 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 18:52:40.231235 24427 mini_ranger.cc:87] Created ranger Postgres database
I20260809 18:52:40.231313 24427 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-admin
2026-08-09 18:52:40,581 [I] DB FLAVOR :POSTGRES
2026-08-09 18:52:40,582 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 18:52:40,582 [I] Checking connection..
2026-08-09 18:52:40,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 18:52:40,828 [I] Checking connection passed.
2026-08-09 18:52:40,828 [I] --------- Verifying version history table ---------
2026-08-09 18:52:40,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 18:52:41,066 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 18:52:41,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 18:52:41,295 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 18:52:41,295 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 18:52:41,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 18:52:41.529 UTC [9292] WARNING: there is no transaction in progress
2026-08-09 18:52:41,537 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 18:52:41,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 18:52:41,792 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 18:52:41,792 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 18:52:41,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 18:52:42,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 18:52:42,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-09 18:52:42,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 18:52:42,736 [I] Table x_portal_user does not exist in database ranger
2026-08-09 18:52:42,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 18:52:42,986 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 18:52:42,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 18:52:43,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 18:52:43,475 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 18:52:43,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 18:52:44.327 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.340 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.354 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.366 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.380 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.397 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.455 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.460 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.472 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.480 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.489 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.501 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.509 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.515 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.520 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:44.525 UTC [9457] WARNING: there is no transaction in progress
2026-08-09 18:52:45,024 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 18:52:45,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 18:52:45,253 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 18:52:45,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 18:52:45,495 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 18:52:45,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 18:52:45,762 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 18:52:45,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 18:52:46,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 18:52:46,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-09 18:52:46,517 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 18:52:46,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 18:52:46,741 [I] DB_PATCHES have already been applied
I20260809 18:52:46.746907 24427 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-admin:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskUXKsiJ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 18:52:46.746966 24427 mini_ranger.cc:193] Using host: 127.23.218.212
I20260809 18:52:46.750329 24427 mini_ranger.cc:227] Ranger admin URL: http://127.23.218.212:46755
Aug 09, 2026 6:52:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 6:52:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 6:52:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 6:52:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 6:52:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 6:52:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 6:52:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 18:52:48.003902 24427 mini_ranger.cc:160] Time spent starting Ranger: real 8.134s user 0.000s sys 0.009s
Aug 09, 2026 6:53:08 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 6:53:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 6:53:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 6:53:09 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 6:53:09 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 18:53:10.579365 24427 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/rangerkms_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/rangerkms_127.23.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.218.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.218.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.218.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/HTTP_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.218.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/HTTP_127.23.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 18:53:10 dist-test-slave-6zbq krb5kdc[9134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301590, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 18:53:10.665561 24427 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 18:53:10.676183 24427 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 18:53:11.396785 24427 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 18:53:11.396869 24427 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-kms
I20260809 18:53:11.396885 24427 mini_ranger_kms.cc:227] Using postgres at 127.23.218.212:46795
2026-08-09 18:53:12,185 [I] DB FLAVOR :POSTGRES
2026-08-09 18:53:12,185 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 18:53:12,185 [I] Checking connection
2026-08-09 18:53:12,185 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 18:53:12,489 [I] connection success
2026-08-09 18:53:12,489 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 18:53:12,489 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 18:53:12,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 18:53:12,734 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 18:53:12,734 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 18:53:12,735 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 18:53:12,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:46795/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 18:53:12,976 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 18:53:13.411085 24427 mini_ranger_kms.cc:310] Created kms service
I20260809 18:53:13.566316 24427 mini_ranger_kms.cc:326] Created kudu user
I20260809 18:53:13.604959 24427 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 18:53:13 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301590, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:53:13.972543 24427 mini_ranger_kms.cc:384] Added ranger policy
I20260809 18:53:13.972646 24427 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-kms:/tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 18:53:13.972672 24427 mini_ranger_kms.cc:240] Using host: 127.23.218.212
I20260809 18:53:13.977418 24427 mini_ranger_kms.cc:276] Ranger KMS PID: 9730
I20260809 18:53:13.977511 24427 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.218.212:37003
Aug 09, 2026 6:53:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 6:53:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 6:53:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 6:53:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 6:53:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 6:53:15 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 18:53:16 dist-test-slave-6zbq krb5kdc[9134](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786301596, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.218.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:53:16.499042 24427 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.833s user 0.008s sys 0.689s
I20260809 18:53:16.499161 24427 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 18:53:16.499239 24427 mini_ranger_kms.cc:401] 127.23.218.212:37003/kms/v1/keys
Aug 09 18:53:16 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301596, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.218.212@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
Aug 09 18:53:17 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301596, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.218.212@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
Aug 09 18:53:19 dist-test-slave-6zbq krb5kdc[9134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:53:19.651089 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:41421
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:41421
--ranger_config_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:37003/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:53:19.755997 9805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:53:19.756194 9805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:53:19.756224 9805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:53:19.757360 9805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:53:19.757400 9805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:53:19.757419 9805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:53:19.757434 9805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:53:19.757450 9805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:53:19.759084 9805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.218.212:37003/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:41421
--ranger_config_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:41421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:53:19.759315 9805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:53:19.759511 9805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:53:19.764436 9821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:53:19.764634 9819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:53:19.764691 9818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:53:19.764501 9805 server_base.cc:1034] running on GCE node
I20260809 18:53:19.765009 9805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:53:19.765332 9805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:53:19.766486 9805 hybrid_clock.cc:648] HybridClock initialized: now 1786301599766468 us; error 35 us; skew 500 ppm
Aug 09 18:53:19 dist-test-slave-6zbq krb5kdc[9134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301599, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:53:19.770143 9805 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:53:19.770699 9805 webserver.cc:460] Webserver started at http://127.23.218.254:32879/ using document root <none> and password file <none>
I20260809 18:53:19.770936 9805 fs_manager.cc:359] Metadata directory not provided
I20260809 18:53:19.770990 9805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:53:19.771131 9805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 18:53:19 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301599, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:53:19.827817 9805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "5323965218a44344ad10d30c73f50d21"
format_stamp: "Formatted at 2026-08-09 18:53:19 on dist-test-slave-6zbq"
server_key: "b6a2f53adb6c1bfa90ea80e84ff10dbc"
server_key_iv: "56231f41dfa332a71654c0a7f68757bc"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:19.828342 9805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "5323965218a44344ad10d30c73f50d21"
format_stamp: "Formatted at 2026-08-09 18:53:19 on dist-test-slave-6zbq"
server_key: "b6a2f53adb6c1bfa90ea80e84ff10dbc"
server_key_iv: "56231f41dfa332a71654c0a7f68757bc"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:19.830067 9805 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 18:53:19.845592 9829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:19.845794 9805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 18:53:19.845897 9805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "5323965218a44344ad10d30c73f50d21"
format_stamp: "Formatted at 2026-08-09 18:53:19 on dist-test-slave-6zbq"
server_key: "b6a2f53adb6c1bfa90ea80e84ff10dbc"
server_key_iv: "56231f41dfa332a71654c0a7f68757bc"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:19.845995 9805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:53:19.876763 9805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:53:19.878659 9805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:53:19.878806 9805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:53:19.888998 9805 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:41421
I20260809 18:53:19.889050 9905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:41421 every 8 connection(s)
I20260809 18:53:19.889396 9805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:53:19.890301 9906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:53:19.893735 9906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21: Bootstrap starting.
I20260809 18:53:19.894843 9906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21: Neither blocks nor log segments found. Creating new log.
I20260809 18:53:19.895609 9906 log.cc:824] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21: Log is configured to *not* fsync() on all Append() calls
I20260809 18:53:19.896486 9906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21: No bootstrap required, opened a new log
I20260809 18:53:19.897662 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 9805
I20260809 18:53:19.897769 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:53:19.897922 24427 external_mini_cluster.cc:1428] Setting key 9c88df10f14631d0bac0aac265db2796
I20260809 18:53:19.900663 9906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5323965218a44344ad10d30c73f50d21" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41421 } }
I20260809 18:53:19.900805 9906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:53:19.900837 9906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5323965218a44344ad10d30c73f50d21, State: Initialized, Role: FOLLOWER
I20260809 18:53:19.900992 9906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5323965218a44344ad10d30c73f50d21" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41421 } }
I20260809 18:53:19.901096 9906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:53:19.901137 9906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:53:19.901188 9906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:53:19.902081 9906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5323965218a44344ad10d30c73f50d21" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41421 } }
I20260809 18:53:19.902181 9906 leader_election.cc:302] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5323965218a44344ad10d30c73f50d21; no voters:
I20260809 18:53:19.902460 9906 leader_election.cc:288] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:53:19.902667 9916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:53:19.902885 9906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:53:19.902988 9916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [term 1 LEADER]: Becoming Leader. State: Replica: 5323965218a44344ad10d30c73f50d21, State: Running, Role: LEADER
I20260809 18:53:19.903085 9916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5323965218a44344ad10d30c73f50d21" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41421 } }
I20260809 18:53:19.904603 9916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5323965218a44344ad10d30c73f50d21. Latest consensus state: current_term: 1 leader_uuid: "5323965218a44344ad10d30c73f50d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5323965218a44344ad10d30c73f50d21" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41421 } } }
I20260809 18:53:19.905066 9918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5323965218a44344ad10d30c73f50d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5323965218a44344ad10d30c73f50d21" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41421 } } }
I20260809 18:53:19.906206 9916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [sys.catalog]: This master's current role is: LEADER
I20260809 18:53:19.906244 9918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21 [sys.catalog]: This master's current role is: LEADER
I20260809 18:53:19.905706 9906 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 18:53:19.906623 9932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 18:53:19 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:53:19.911677 9932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:53:19.914330 9932 catalog_manager.cc:1347] Generated new cluster ID: f93bedfc6a724f4dbd3b6d395bc5cf90
I20260809 18:53:19.914400 9932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:53:19.915473 9915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:19.898449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59064 (local address 127.23.218.254:41421)
0809 18:53:19.898830 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:19.898838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:19.898864 (+ 26us) server_negotiation.cc:407] Connection header received
0809 18:53:19.900670 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:19.900677 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:19.900727 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:19.900844 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:53:19.901764 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:19.903079 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:19.904112 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:19.904643 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:19.904749 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:19.909308 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:53:19.909414 (+ 106us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:53:19.909414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:53:19.909748 (+ 334us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:53:19.913746 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:19.914183 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:19.914186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:19.914186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:19.914282 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:19.914526 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:19.914527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:19.914527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:19.914818 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:53:19.914948 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:19.915073 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 18:53:19.915214 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":283,"threads_started":1}
I20260809 18:53:19.927630 9932 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:53:19.928393 9932 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:53:19.947706 9932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21: Generated new TSK 0
I20260809 18:53:19.947958 9932 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:53:20.827067 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:37003/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:41421
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:53:20.970399 9968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:53:20.970650 9968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:53:20.970685 9968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:53:20.972476 9968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:53:20.972527 9968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:53:20.972564 9968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:53:20.974938 9968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.218.212:37003/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:41421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:53:20.975180 9968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:53:20.975455 9968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:53:20.982373 9977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:53:20.982473 9982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:53:20.982656 9968 server_base.cc:1034] running on GCE node
W20260809 18:53:20.982640 9979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:53:20.982954 9968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:53:20.983182 9968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:53:20.984326 9968 hybrid_clock.cc:648] HybridClock initialized: now 1786301600984310 us; error 32 us; skew 500 ppm
Aug 09 18:53:20 dist-test-slave-6zbq krb5kdc[9134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301600, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:53:20.987267 9968 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:53:20.987679 9968 webserver.cc:460] Webserver started at http://127.23.218.193:33897/ using document root <none> and password file <none>
I20260809 18:53:20.987880 9968 fs_manager.cc:359] Metadata directory not provided
I20260809 18:53:20.987926 9968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:53:20.988042 9968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 18:53:20 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301600, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:53:21.001958 9968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "654c17e3db924206a9516b52fdfbf7bb"
format_stamp: "Formatted at 2026-08-09 18:53:20 on dist-test-slave-6zbq"
server_key: "74064ae76bb8e30cf5504e3da4c275ba"
server_key_iv: "26da643662f799c9b433322dd32738ef"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:21.002554 9968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "654c17e3db924206a9516b52fdfbf7bb"
format_stamp: "Formatted at 2026-08-09 18:53:20 on dist-test-slave-6zbq"
server_key: "74064ae76bb8e30cf5504e3da4c275ba"
server_key_iv: "26da643662f799c9b433322dd32738ef"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:21.004684 9968 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 18:53:21.022969 9994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:21.023185 9968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 18:53:21.023309 9968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "654c17e3db924206a9516b52fdfbf7bb"
format_stamp: "Formatted at 2026-08-09 18:53:20 on dist-test-slave-6zbq"
server_key: "74064ae76bb8e30cf5504e3da4c275ba"
server_key_iv: "26da643662f799c9b433322dd32738ef"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:21.023403 9968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:53:21.052148 9968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:53:21.054502 9968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:53:21.054653 9968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:53:21.054956 9968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:53:21.055369 9968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:53:21.055416 9968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:21.055444 9968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:53:21.055496 9968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:21.064694 9968 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:32851
I20260809 18:53:21.065174 9968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:53:21.072496 10171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:32851 every 8 connection(s)
I20260809 18:53:21.074935 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 9968
I20260809 18:53:21.075040 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:53:21.075214 24427 external_mini_cluster.cc:1428] Setting key 5e2c60cd4192c926df7a64178ee85f90
Aug 09 18:53:21 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301600, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:53:21.085536 10177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.073148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41421 (local address 127.23.218.193:50391)
0809 18:53:21.073451 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.073495 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.074182 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.074648 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.074654 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.075654 (+ 1000us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:53:21.075978 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.075986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.077115 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.077116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.078550 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.078553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.079061 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.079062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.079228 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.079770 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:53:21.079779 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:53:21.081455 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:53:21.083922 (+ 2467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:53:21.083929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:53:21.083931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:53:21.084208 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:53:21.084450 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:53:21.084450 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:53:21.084451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:53:21.084579 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:53:21.084872 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:53:21.084878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:53:21.085029 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.085133 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.085290 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":104,"threads_started":1}
I20260809 18:53:21.085839 10175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.073168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:50391 (local address 127.23.218.254:41421)
0809 18:53:21.074406 (+ 1238us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.074408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.074417 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:53:21.074443 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.074444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.074459 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.075707 (+ 1248us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:53:21.076076 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.076999 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.078655 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.078953 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.079466 (+ 513us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.081555 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:53:21.081556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:53:21.081557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:53:21.081564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:53:21.083821 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:21.084314 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:21.084315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:21.084315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:21.084381 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:21.084651 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:21.084653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:21.084653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:21.084785 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:53:21.084845 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.085666 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.085748 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1214,"thread_start_us":64,"threads_started":1}
I20260809 18:53:21.086287 10172 heartbeater.cc:347] Connected to a master server at 127.23.218.254:41421
I20260809 18:53:21.086431 10172 heartbeater.cc:464] Registering TS with master...
I20260809 18:53:21.086648 10172 heartbeater.cc:511] Master 127.23.218.254:41421 requested a full tablet report, sending...
I20260809 18:53:21.087215 9855 ts_manager.cc:194] Registered new tserver with Master: 654c17e3db924206a9516b52fdfbf7bb (127.23.218.193:32851)
I20260809 18:53:21.088069 9855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:50391
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:53:21.124977 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:37003/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:41421
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:53:21.293329 10180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:53:21.293708 10180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:53:21.293800 10180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:53:21.295688 10180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:53:21.295890 10180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:53:21.295977 10180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:53:21.298520 10180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.218.212:37003/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:41421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:53:21.298947 10180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:53:21.299264 10180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:53:21.305434 10196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:53:21.305450 10197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:53:21.305915 10180 server_base.cc:1034] running on GCE node
W20260809 18:53:21.306360 10199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:53:21.306627 10180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:53:21.306870 10180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:53:21.308073 10180 hybrid_clock.cc:648] HybridClock initialized: now 1786301601308079 us; error 43 us; skew 500 ppm
Aug 09 18:53:21 dist-test-slave-6zbq krb5kdc[9134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:53:21.310460 10180 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:53:21.310904 10180 webserver.cc:460] Webserver started at http://127.23.218.194:36309/ using document root <none> and password file <none>
I20260809 18:53:21.311111 10180 fs_manager.cc:359] Metadata directory not provided
I20260809 18:53:21.311159 10180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:53:21.311290 10180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 18:53:21 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
Aug 09 18:53:21 dist-test-slave-6zbq krb5kdc[9134](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786301601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:53:21.324581 10180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "fbae6d64234b4485a1bbc6d72ff9ade7"
format_stamp: "Formatted at 2026-08-09 18:53:21 on dist-test-slave-6zbq"
server_key: "69dbd25d5aa88120455d6644e7a92914"
server_key_iv: "0dbcad8e55b7530b86f6c38d7b442ed7"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:21.325101 10180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "fbae6d64234b4485a1bbc6d72ff9ade7"
format_stamp: "Formatted at 2026-08-09 18:53:21 on dist-test-slave-6zbq"
server_key: "69dbd25d5aa88120455d6644e7a92914"
server_key_iv: "0dbcad8e55b7530b86f6c38d7b442ed7"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:21.328027 10180 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260809 18:53:21.345208 10211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:21.345427 10180 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260809 18:53:21.345510 10180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "fbae6d64234b4485a1bbc6d72ff9ade7"
format_stamp: "Formatted at 2026-08-09 18:53:21 on dist-test-slave-6zbq"
server_key: "69dbd25d5aa88120455d6644e7a92914"
server_key_iv: "0dbcad8e55b7530b86f6c38d7b442ed7"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:21.345582 10180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:53:21.365278 10180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:53:21.369786 10180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:53:21.370024 10180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:53:21.370324 10180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:53:21.370800 10180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:53:21.370885 10180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:21.370954 10180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:53:21.371004 10180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:21.383464 10180 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:40481
I20260809 18:53:21.383935 10180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:53:21.384413 10388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:40481 every 8 connection(s)
I20260809 18:53:21.390327 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 10180
I20260809 18:53:21.390446 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:53:21.390609 24427 external_mini_cluster.cc:1428] Setting key 43f1f8777082ab0a6f774c6ecd83033e
Aug 09 18:53:21 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:53:21.399284 10175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.388384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:47997 (local address 127.23.218.254:41421)
0809 18:53:21.388430 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.388432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.388694 (+ 262us) server_negotiation.cc:407] Connection header received
0809 18:53:21.389553 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.389555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.389569 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.389626 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:53:21.390457 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.391262 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.392883 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.393262 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.393308 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.396083 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:53:21.396085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:53:21.396085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:53:21.396104 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:53:21.397665 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:21.398123 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:21.398125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:21.398125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:21.398187 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:21.398509 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:21.398510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:21.398511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:21.398653 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:53:21.398703 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.399084 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.399190 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:53:21.399827 10394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.388028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41421 (local address 127.23.218.194:47997)
0809 18:53:21.388526 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.388580 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.389427 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.389714 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.389719 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.390016 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:53:21.390314 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.390322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.391381 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.391382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.392776 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.392779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.393401 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.393402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.393567 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.394154 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:53:21.394165 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:53:21.395978 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:53:21.397751 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:53:21.397758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:53:21.397760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:53:21.398024 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:53:21.398253 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:53:21.398254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:53:21.398254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:53:21.398423 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:53:21.398756 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:53:21.398763 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:53:21.398927 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.399407 (+ 480us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.399558 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":142,"threads_started":1}
I20260809 18:53:21.400530 10389 heartbeater.cc:347] Connected to a master server at 127.23.218.254:41421
I20260809 18:53:21.400633 10389 heartbeater.cc:464] Registering TS with master...
I20260809 18:53:21.400812 10389 heartbeater.cc:511] Master 127.23.218.254:41421 requested a full tablet report, sending...
I20260809 18:53:21.401253 9855 ts_manager.cc:194] Registered new tserver with Master: fbae6d64234b4485a1bbc6d72ff9ade7 (127.23.218.194:40481)
I20260809 18:53:21.401714 9855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:47997
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:53:21.430202 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:37003/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:41421
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:53:21.586254 10402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:53:21.586580 10402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:53:21.586695 10402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:53:21.588375 10402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:53:21.588434 10402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:53:21.588475 10402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:53:21.590849 10402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.218.212:37003/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:41421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:53:21.591089 10402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:53:21.591428 10402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:53:21.599300 10414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:53:21.599437 10411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:53:21.599792 10410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:53:21.600294 10402 server_base.cc:1034] running on GCE node
I20260809 18:53:21.600507 10402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:53:21.600795 10402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:53:21.601981 10402 hybrid_clock.cc:648] HybridClock initialized: now 1786301601601953 us; error 40 us; skew 500 ppm
Aug 09 18:53:21 dist-test-slave-6zbq krb5kdc[9134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:53:21.605861 10402 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:53:21.606436 10402 webserver.cc:460] Webserver started at http://127.23.218.195:39391/ using document root <none> and password file <none>
I20260809 18:53:21.606702 10402 fs_manager.cc:359] Metadata directory not provided
I20260809 18:53:21.606776 10402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:53:21.606917 10402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 18:53:21 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:53:21.629976 10402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "8cbf722bb1c94563937db0ce5b1f8a5e"
format_stamp: "Formatted at 2026-08-09 18:53:21 on dist-test-slave-6zbq"
server_key: "f32e60fc117a8cce2717e0ebf7211a0c"
server_key_iv: "5284cfb42a115de5a81fe3b07e8d19ff"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:21.630481 10402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "8cbf722bb1c94563937db0ce5b1f8a5e"
format_stamp: "Formatted at 2026-08-09 18:53:21 on dist-test-slave-6zbq"
server_key: "f32e60fc117a8cce2717e0ebf7211a0c"
server_key_iv: "5284cfb42a115de5a81fe3b07e8d19ff"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:21.632685 10402 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 18:53:21.650013 10427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:21.650182 10402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 18:53:21.650276 10402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "8cbf722bb1c94563937db0ce5b1f8a5e"
format_stamp: "Formatted at 2026-08-09 18:53:21 on dist-test-slave-6zbq"
server_key: "f32e60fc117a8cce2717e0ebf7211a0c"
server_key_iv: "5284cfb42a115de5a81fe3b07e8d19ff"
server_key_version: "kuduclusterkey@0"
I20260809 18:53:21.650396 10402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:53:21.674088 10402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:53:21.675904 10402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:53:21.676038 10402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:53:21.676237 10402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:53:21.676589 10402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:53:21.676645 10402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:21.676676 10402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:53:21.676697 10402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:53:21.693306 10402 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:39913
I20260809 18:53:21.693462 10601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:39913 every 8 connection(s)
I20260809 18:53:21.693794 10402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:53:21.697791 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 10402
I20260809 18:53:21.697880 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:53:21.698047 24427 external_mini_cluster.cc:1428] Setting key d9044ad63b50a6e40d3dcac1dd0b3026
Aug 09 18:53:21 dist-test-slave-6zbq krb5kdc[9134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:53:21.704412 10175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.694624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:46863 (local address 127.23.218.254:41421)
0809 18:53:21.694690 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.694692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.694936 (+ 244us) server_negotiation.cc:407] Connection header received
0809 18:53:21.695856 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.695857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.695867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.695908 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:53:21.696854 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.697336 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.698883 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.699092 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.699146 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.701693 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:53:21.701694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:53:21.701695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:53:21.701702 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:53:21.702953 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:21.703399 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:21.703399 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:21.703400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:21.703464 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:21.703755 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:21.703756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:21.703756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:21.703856 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:53:21.703905 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.704246 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.704327 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 18:53:21.704646 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.694585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:41421 (local address 127.23.218.195:46863)
0809 18:53:21.694790 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.694840 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.695754 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.695987 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.695992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.696383 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:53:21.696760 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.696769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.697426 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.697428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.698790 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.698794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.699175 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.699176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.699392 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.699902 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:53:21.699913 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:53:21.701606 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:53:21.703023 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:53:21.703030 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:53:21.703032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:53:21.703334 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:53:21.703508 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:53:21.703509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:53:21.703509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:53:21.703694 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:53:21.703914 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:53:21.703920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:53:21.704110 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.704229 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.704385 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260809 18:53:21.705397 10602 heartbeater.cc:347] Connected to a master server at 127.23.218.254:41421
I20260809 18:53:21.705497 10602 heartbeater.cc:464] Registering TS with master...
I20260809 18:53:21.705682 10602 heartbeater.cc:511] Master 127.23.218.254:41421 requested a full tablet report, sending...
I20260809 18:53:21.706064 9855 ts_manager.cc:194] Registered new tserver with Master: 8cbf722bb1c94563937db0ce5b1f8a5e (127.23.218.195:39913)
I20260809 18:53:21.706480 9855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:46863
I20260809 18:53:21.709677 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:53:21.716710 10175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.711109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59066 (local address 127.23.218.254:41421)
0809 18:53:21.711151 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.711152 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.711362 (+ 210us) server_negotiation.cc:407] Connection header received
0809 18:53:21.711419 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.711420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.711429 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.711473 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:53:21.712070 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.712529 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.713213 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.713407 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.713458 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.714580 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:53:21.714581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:53:21.714581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:53:21.714588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:53:21.715765 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:21.716048 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:21.716049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:21.716049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:21.716113 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:53:21.716275 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:53:21.716276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:53:21.716276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:53:21.716377 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:53:21.716457 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.716562 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.716630 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:53:21.718107 9855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:53:21.719879 9855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:53:21.726455 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.722591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:32851 (local address 127.0.0.1:51486)
0809 18:53:21.723043 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.723081 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.723152 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.723503 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.723506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.723508 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:53:21.723570 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.723571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.724519 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.724520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.725364 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.725367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.726152 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.726154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.726231 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.726246 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.726309 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.726379 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":137,"threads_started":1}
I20260809 18:53:21.726497 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.723127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.23.218.193:32851)
0809 18:53:21.723327 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.723331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.723340 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:53:21.723371 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.723373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.723389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.723457 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:53:21.723625 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.724441 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.725472 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.726075 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.726144 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.726203 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.726329 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.726391 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260809 18:53:21.727452 10076 tablet_service.cc:1467] Processing CreateTablet for tablet 445b74a644cd4306932ecc49054a6f39 (DEFAULT_TABLE table=test-table [id=c6fec7a21daa4b18a473872f4080179b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:53:21.727841 10076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 445b74a644cd4306932ecc49054a6f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:53:21.731317 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.723001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36128 (local address 127.23.218.194:40481)
0809 18:53:21.723316 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.723322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.726794 (+ 3472us) server_negotiation.cc:407] Connection header received
0809 18:53:21.727010 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.727012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.727029 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.727115 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:53:21.727328 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.728814 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.730135 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.730850 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.730911 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.730965 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.731198 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.731237 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260809 18:53:21.731479 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.722609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:39913 (local address 127.0.0.1:34450)
0809 18:53:21.726501 (+ 3892us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.726513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.726579 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.727213 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.727214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.727216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:53:21.727267 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.727269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.728787 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.728788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.729940 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.729945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.731260 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.731261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.731328 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.731333 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.731365 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.731404 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3847,"thread_start_us":95,"threads_started":1}
I20260809 18:53:21.731698 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.722845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:40481 (local address 127.0.0.1:36128)
0809 18:53:21.726726 (+ 3881us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.726736 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.726793 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.727207 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.727208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.727210 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:53:21.727248 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.727249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.728916 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.728916 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.729987 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.729991 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.731028 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.731029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.731088 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.731091 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.731584 (+ 493us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.731629 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3839,"thread_start_us":129,"threads_started":1}
I20260809 18:53:21.732302 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.722726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.23.218.195:39913)
0809 18:53:21.727005 (+ 4279us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.727010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.727021 (+ 11us) server_negotiation.cc:407] Connection header received
0809 18:53:21.727080 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.727082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.727098 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.727343 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:53:21.727458 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.728679 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.730181 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.731155 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.732033 (+ 878us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.732102 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.732131 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.732170 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4214,"thread_start_us":122,"threads_started":1}
I20260809 18:53:21.732987 10503 tablet_service.cc:1467] Processing CreateTablet for tablet 445b74a644cd4306932ecc49054a6f39 (DEFAULT_TABLE table=test-table [id=c6fec7a21daa4b18a473872f4080179b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:53:21.733067 10281 tablet_service.cc:1467] Processing CreateTablet for tablet 445b74a644cd4306932ecc49054a6f39 (DEFAULT_TABLE table=test-table [id=c6fec7a21daa4b18a473872f4080179b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:53:21.733323 10503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 445b74a644cd4306932ecc49054a6f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:53:21.733399 10281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 445b74a644cd4306932ecc49054a6f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:53:21.738854 10631 tablet_bootstrap.cc:492] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7: Bootstrap starting.
I20260809 18:53:21.740253 10631 tablet_bootstrap.cc:654] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7: Neither blocks nor log segments found. Creating new log.
I20260809 18:53:21.740854 10631 log.cc:824] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7: Log is configured to *not* fsync() on all Append() calls
I20260809 18:53:21.741760 10631 tablet_bootstrap.cc:492] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7: No bootstrap required, opened a new log
I20260809 18:53:21.741819 10631 ts_tablet_manager.cc:1397] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:53:21.741894 10629 tablet_bootstrap.cc:492] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb: Bootstrap starting.
I20260809 18:53:21.743484 10629 tablet_bootstrap.cc:654] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb: Neither blocks nor log segments found. Creating new log.
I20260809 18:53:21.744266 10629 log.cc:824] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb: Log is configured to *not* fsync() on all Append() calls
I20260809 18:53:21.745560 10631 raft_consensus.cc:348] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.745738 10631 raft_consensus.cc:374] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:53:21.745777 10631 raft_consensus.cc:729] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbae6d64234b4485a1bbc6d72ff9ade7, State: Initialized, Role: FOLLOWER
I20260809 18:53:21.746069 10631 consensus_queue.cc:260] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.746420 10631 ts_tablet_manager.cc:1428] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 18:53:21.746537 10389 heartbeater.cc:503] Master 127.23.218.254:41421 was elected leader, sending a full tablet report...
I20260809 18:53:21.748708 10629 tablet_bootstrap.cc:492] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb: No bootstrap required, opened a new log
I20260809 18:53:21.748786 10629 ts_tablet_manager.cc:1397] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.003s
I20260809 18:53:21.753659 10629 raft_consensus.cc:348] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.753868 10629 raft_consensus.cc:374] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:53:21.753925 10629 raft_consensus.cc:729] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 654c17e3db924206a9516b52fdfbf7bb, State: Initialized, Role: FOLLOWER
I20260809 18:53:21.754149 10629 consensus_queue.cc:260] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.754379 10630 tablet_bootstrap.cc:492] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: Bootstrap starting.
I20260809 18:53:21.754487 10629 ts_tablet_manager.cc:1428] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260809 18:53:21.754626 10172 heartbeater.cc:503] Master 127.23.218.254:41421 was elected leader, sending a full tablet report...
I20260809 18:53:21.755899 10630 tablet_bootstrap.cc:654] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: Neither blocks nor log segments found. Creating new log.
I20260809 18:53:21.757007 10630 log.cc:824] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: Log is configured to *not* fsync() on all Append() calls
I20260809 18:53:21.757820 10630 tablet_bootstrap.cc:492] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: No bootstrap required, opened a new log
I20260809 18:53:21.757895 10630 ts_tablet_manager.cc:1397] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 18:53:21.761521 10630 raft_consensus.cc:348] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.761684 10630 raft_consensus.cc:374] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:53:21.761723 10630 raft_consensus.cc:729] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cbf722bb1c94563937db0ce5b1f8a5e, State: Initialized, Role: FOLLOWER
I20260809 18:53:21.761861 10630 consensus_queue.cc:260] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.762181 10630 ts_tablet_manager.cc:1428] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260809 18:53:21.762297 10602 heartbeater.cc:503] Master 127.23.218.254:41421 was elected leader, sending a full tablet report...
I20260809 18:53:21.765430 10635 raft_consensus.cc:482] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:53:21.765595 10635 raft_consensus.cc:504] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.765945 10635 leader_election.cc:288] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cbf722bb1c94563937db0ce5b1f8a5e (127.23.218.195:39913), fbae6d64234b4485a1bbc6d72ff9ade7 (127.23.218.194:40481)
I20260809 18:53:21.770630 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.766399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:37807 (local address 127.23.218.195:39913)
0809 18:53:21.766439 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.766441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.766588 (+ 147us) server_negotiation.cc:407] Connection header received
0809 18:53:21.766771 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.766772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.766782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.766809 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:53:21.767229 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.767995 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.769610 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.770178 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.770250 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.770259 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.770530 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.770573 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:53:21.771037 10644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.766152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:39913 (local address 127.23.218.193:37807)
0809 18:53:21.766517 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.766529 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.766685 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.766872 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.766873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.766875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:53:21.767146 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.767148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.768114 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.768115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.769517 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.769521 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.770267 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.770268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.770427 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.770433 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.770872 (+ 439us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.770917 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":162,"threads_started":1}
I20260809 18:53:21.771656 10505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b74a644cd4306932ecc49054a6f39" candidate_uuid: "654c17e3db924206a9516b52fdfbf7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" is_pre_election: true
I20260809 18:53:21.771785 10505 raft_consensus.cc:2454] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654c17e3db924206a9516b52fdfbf7bb in term 0.
I20260809 18:53:21.772142 9999 leader_election.cc:302] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654c17e3db924206a9516b52fdfbf7bb, 8cbf722bb1c94563937db0ce5b1f8a5e; no voters:
I20260809 18:53:21.772267 10635 raft_consensus.cc:2781] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:53:21.772342 10635 raft_consensus.cc:482] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:53:21.772369 10635 raft_consensus.cc:3037] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:53:21.772416 10639 raft_consensus.cc:482] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:53:21.772467 10639 raft_consensus.cc:504] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.772886 10639 leader_election.cc:288] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 654c17e3db924206a9516b52fdfbf7bb (127.23.218.193:32851), fbae6d64234b4485a1bbc6d72ff9ade7 (127.23.218.194:40481)
I20260809 18:53:21.773524 10635 raft_consensus.cc:504] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.773633 10635 leader_election.cc:288] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [CANDIDATE]: Term 1 election: Requested vote from peers 8cbf722bb1c94563937db0ce5b1f8a5e (127.23.218.195:39913), fbae6d64234b4485a1bbc6d72ff9ade7 (127.23.218.194:40481)
I20260809 18:53:21.773967 10505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b74a644cd4306932ecc49054a6f39" candidate_uuid: "654c17e3db924206a9516b52fdfbf7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e"
I20260809 18:53:21.774035 10505 raft_consensus.cc:3037] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:53:21.775295 10505 raft_consensus.cc:2454] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654c17e3db924206a9516b52fdfbf7bb in term 1.
I20260809 18:53:21.775501 9999 leader_election.cc:302] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654c17e3db924206a9516b52fdfbf7bb, 8cbf722bb1c94563937db0ce5b1f8a5e; no voters:
I20260809 18:53:21.775609 10635 raft_consensus.cc:2781] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:53:21.775728 10635 raft_consensus.cc:686] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [term 1 LEADER]: Becoming Leader. State: Replica: 654c17e3db924206a9516b52fdfbf7bb, State: Running, Role: LEADER
I20260809 18:53:21.775804 10635 consensus_queue.cc:237] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } }
I20260809 18:53:21.776572 9855 catalog_manager.cc:5495] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 654c17e3db924206a9516b52fdfbf7bb (127.23.218.193). New cstate: current_term: 1 leader_uuid: "654c17e3db924206a9516b52fdfbf7bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654c17e3db924206a9516b52fdfbf7bb" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 32851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 } health_report { overall_health: UNKNOWN } } }
I20260809 18:53:21.777376 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.773057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:32851 (local address 127.23.218.195:50955)
0809 18:53:21.773376 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.773386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.773427 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.774011 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.774012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.774014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:53:21.774066 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.774067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.775224 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.775225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.776414 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.776417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.777139 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.777140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.777229 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.777232 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.777277 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.777312 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303}
I20260809 18:53:21.778072 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.773835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:50955 (local address 127.23.218.193:32851)
0809 18:53:21.773873 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.773874 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.773882 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:53:21.773901 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.773901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.773910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.773971 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:53:21.774133 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.774879 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.776511 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.777040 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.777102 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.777109 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.777976 (+ 867us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.778014 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:53:21.778496 10104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b74a644cd4306932ecc49054a6f39" candidate_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "654c17e3db924206a9516b52fdfbf7bb" is_pre_election: true
I20260809 18:53:21.780951 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.766859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:33059 (local address 127.23.218.194:40481)
0809 18:53:21.766899 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.766901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.770573 (+ 3672us) server_negotiation.cc:407] Connection header received
0809 18:53:21.770664 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.770665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.770680 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.770708 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:53:21.771027 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.771770 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.779947 (+ 8177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.780498 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.780543 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.780550 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.780844 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.780887 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:53:21.783495 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.766564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:40481 (local address 127.23.218.193:33059)
0809 18:53:21.770509 (+ 3945us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.770520 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.770599 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.770767 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.770768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.770770 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:53:21.770935 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.770937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.771919 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.771920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.779843 (+ 7923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.779847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.780605 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.780606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.780748 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.780752 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.783249 (+ 2497us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.783290 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3853,"thread_start_us":144,"threads_started":1}
I20260809 18:53:21.783999 10317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b74a644cd4306932ecc49054a6f39" candidate_uuid: "654c17e3db924206a9516b52fdfbf7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" is_pre_election: true
I20260809 18:53:21.784145 10317 raft_consensus.cc:2454] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654c17e3db924206a9516b52fdfbf7bb in term 0.
I20260809 18:53:21.784271 10318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b74a644cd4306932ecc49054a6f39" candidate_uuid: "654c17e3db924206a9516b52fdfbf7bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7"
I20260809 18:53:21.784379 10318 raft_consensus.cc:3037] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:53:21.785465 10318 raft_consensus.cc:2454] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654c17e3db924206a9516b52fdfbf7bb in term 1.
I20260809 18:53:21.785907 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.773152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:40481 (local address 127.23.218.195:55793)
0809 18:53:21.777442 (+ 4290us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.777451 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:53:21.777481 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:53:21.781180 (+ 3699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:53:21.781181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:53:21.781183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:53:21.781212 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.781213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.782082 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.782083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.783161 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:53:21.783164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.785678 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.785679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:53:21.785745 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.785748 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:53:21.785810 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 18:53:21.785844 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4281,"thread_start_us":145,"threads_started":1}
I20260809 18:53:21.785924 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.773527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:55793 (local address 127.23.218.194:40481)
0809 18:53:21.781024 (+ 7497us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.781026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.781031 (+ 5us) server_negotiation.cc:407] Connection header received
0809 18:53:21.781047 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.781048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.781056 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.781114 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:53:21.781273 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.781979 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.783255 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.783779 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.783852 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.783858 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.785822 (+ 1964us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.785862 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7484,"thread_start_us":155,"threads_started":1}
I20260809 18:53:21.786105 10318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "445b74a644cd4306932ecc49054a6f39" candidate_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" is_pre_election: true
I20260809 18:53:21.786185 10318 raft_consensus.cc:2379] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8cbf722bb1c94563937db0ce5b1f8a5e in current term 1: Already voted for candidate 654c17e3db924206a9516b52fdfbf7bb in this term.
I20260809 18:53:21.786415 10431 leader_election.cc:302] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8cbf722bb1c94563937db0ce5b1f8a5e; no voters: 654c17e3db924206a9516b52fdfbf7bb, fbae6d64234b4485a1bbc6d72ff9ade7
I20260809 18:53:21.786583 10639 raft_consensus.cc:2726] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 18:53:21.792210 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.789092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51494 (local address 127.23.218.193:32851)
0809 18:53:21.789124 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:53:21.789125 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:53:21.789132 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:53:21.789159 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:53:21.789160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:53:21.789177 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:53:21.789232 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:53:21.789409 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.790031 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.791102 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:53:21.791384 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:53:21.791458 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:53:21.791655 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:53:21.791776 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:53:21.791874 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 18:53:21.792034 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 18:53:21.792054 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:53:21.794549 10318 raft_consensus.cc:1261] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 1 FOLLOWER]: Refusing update from remote peer 654c17e3db924206a9516b52fdfbf7bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:53:21.794584 10505 raft_consensus.cc:1261] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 1 FOLLOWER]: Refusing update from remote peer 654c17e3db924206a9516b52fdfbf7bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:53:21.794847 10635 consensus_queue.cc:1035] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cbf722bb1c94563937db0ce5b1f8a5e" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 39913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:53:21.794997 10635 consensus_queue.cc:1035] T 445b74a644cd4306932ecc49054a6f39 P 654c17e3db924206a9516b52fdfbf7bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbae6d64234b4485a1bbc6d72ff9ade7" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 40481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:53:21.799292 10658 mvcc.cc:205] Tried to move back new op lower bound from 7316691360947785728 to 7316691360874147840. Current Snapshot: MvccSnapshot[applied={T|T < 7316691360947785728}]
I20260809 18:53:21.799582 10657 mvcc.cc:205] Tried to move back new op lower bound from 7316691360947785728 to 7316691360874147840. Current Snapshot: MvccSnapshot[applied={T|T < 7316691360947785728}]
I20260809 18:53:21.800155 10661 mvcc.cc:205] Tried to move back new op lower bound from 7316691360947785728 to 7316691360874147840. Current Snapshot: MvccSnapshot[applied={T|T < 7316691360947785728}]
I20260809 18:53:21.802178 9855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:53:21.802264 9855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:53:21.803066 9855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5323965218a44344ad10d30c73f50d21: Sending DeleteTablet for 3 replicas of tablet 445b74a644cd4306932ecc49054a6f39
I20260809 18:53:21.803557 10503 tablet_service.cc:1514] Processing DeleteTablet for tablet 445b74a644cd4306932ecc49054a6f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:53:21 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:34450
I20260809 18:53:21.804016 10076 tablet_service.cc:1514] Processing DeleteTablet for tablet 445b74a644cd4306932ecc49054a6f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:53:21 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:51486
I20260809 18:53:21.804219 10672 tablet_replica.cc:330] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: stopping tablet replica
I20260809 18:53:21.804232 10281 tablet_service.cc:1514] Processing DeleteTablet for tablet 445b74a644cd4306932ecc49054a6f39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:53:21 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:36128
I20260809 18:53:21.804313 10672 raft_consensus.cc:2229] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:53:21.804407 10672 raft_consensus.cc:2258] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:53:21.804816 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 9968
I20260809 18:53:21.804828 10672 ts_tablet_manager.cc:1905] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:53:21.809080 10672 ts_tablet_manager.cc:1918] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:53:21.809154 10672 log.cc:1196] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/445b74a644cd4306932ecc49054a6f39
I20260809 18:53:21.809785 10672 ts_tablet_manager.cc:1939] T 445b74a644cd4306932ecc49054a6f39 P 8cbf722bb1c94563937db0ce5b1f8a5e: Deleting consensus metadata
I20260809 18:53:21.810288 9832 catalog_manager.cc:4841] TS 8cbf722bb1c94563937db0ce5b1f8a5e (127.23.218.195:39913): tablet 445b74a644cd4306932ecc49054a6f39 (table test-table [id=c6fec7a21daa4b18a473872f4080179b]) successfully deleted
I20260809 18:53:21.816049 10675 tablet_replica.cc:330] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7: stopping tablet replica
I20260809 18:53:21.816176 10675 raft_consensus.cc:2229] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:53:21.816290 10675 raft_consensus.cc:2258] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 18:53:21.821633 9831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:32851 (error 108)
I20260809 18:53:21.821905 10675 ts_tablet_manager.cc:1905] T 445b74a644cd4306932ecc49054a6f39 P fbae6d64234b4485a1bbc6d72ff9ade7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:53:21.822463 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.822322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:32851 (local address 127.0.0.1:51500)
0809 18:53:21.822390 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.822429 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:32851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 18:53:21.822588 9831 catalog_manager.cc:4584] TS 654c17e3db924206a9516b52fdfbf7bb (127.23.218.193:32851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 445b74a644cd4306932ecc49054a6f39: Network error: Client connection negotiation failed: client connection to 127.23.218.193:32851: connect: Connection refused (error 111)
I20260809 18:53:21.822765 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 10180
I20260809 18:53:21.832749 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 10402
I20260809 18:53:21.832764 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:53:21.832663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:40481 (local address 127.0.0.1:36142)
0809 18:53:21.832705 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 18:53:21.832732 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.194:40481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 18:53:21.840466 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 9805
I20260809 18:53:21.851724 24427 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 18:53:22.196108 24427 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 18:53:22.196208 24427 mini_ranger.cc:66] Stopping Ranger...
I20260809 18:53:22.539554 24427 mini_ranger.cc:68] Stopped Ranger
2026-08-09 18:53:22.539 UTC [9186] LOG: received smart shutdown request
2026-08-09 18:53:22.543 UTC [9186] LOG: background worker "logical replication launcher" (PID 9197) exited with exit code 1
2026-08-09 18:53:22.543 UTC [9191] LOG: shutting down
2026-08-09 18:53:22.669 UTC [9186] LOG: database system is shut down
2026-08-09T18:53:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47392 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:53:22 dist-test-slave-6zbq krb5kdc[10694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:53:22 dist-test-slave-6zbq krb5kdc[10694](info): set up 2 sockets
Aug 09 18:53:22 dist-test-slave-6zbq krb5kdc[10694](info): commencing operation
krb5kdc: starting...
W20260809 18:53:23.910523 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:53:23 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:53:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:53:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/rangeradmin_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/rangeradmin_127.23.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/rangerlookup_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/rangerlookup_127.23.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/HTTP_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/HTTP_127.23.218.212@KRBTEST.COM.keytab.
I20260809 18:53:24.052286 24427 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskUXKsiJ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/postgres -l logfile start
2026-08-09 18:53:26.111 UTC [10729] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 18:53:26.111 UTC [10729] LOG: listening on IPv4 address "127.23.218.212", port 48259
2026-08-09 18:53:26.114 UTC [10729] LOG: listening on Unix socket "/tmp/.s.PGSQL.48259"
2026-08-09 18:53:26.132 UTC [10734] LOG: database system was shut down at 2026-08-09 18:53:25 UTC
2026-08-09 18:53:26.135 UTC [10729] LOG: database system is ready to accept connections
I20260809 18:53:27.198936 24427 mini_postgres.cc:96] Postgres bound to 48259
2026-08-09 18:53:27.205 UTC [10749] FATAL: database "slave" does not exist
127.23.218.212:48259 - accepting connections
I20260809 18:53:27.205927 24427 mini_ranger.cc:161] Starting Ranger...
I20260809 18:53:27.219801 24427 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 18:53:27.602954 24427 mini_ranger.cc:87] Created ranger Postgres database
I20260809 18:53:27.603024 24427 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-admin
2026-08-09 18:53:27,874 [I] DB FLAVOR :POSTGRES
2026-08-09 18:53:27,875 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 18:53:27,875 [I] Checking connection..
2026-08-09 18:53:27,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 18:53:28,119 [I] Checking connection passed.
2026-08-09 18:53:28,120 [I] --------- Verifying version history table ---------
2026-08-09 18:53:28,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 18:53:28,366 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 18:53:28,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 18:53:28,604 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 18:53:28,605 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 18:53:28,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 18:53:28.829 UTC [10847] WARNING: there is no transaction in progress
2026-08-09 18:53:28,837 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 18:53:28,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 18:53:29,093 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 18:53:29,093 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 18:53:29,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 18:53:29,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 18:53:29,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6zbq.c.gcp-upstream.internal','N') ;"
2026-08-09 18:53:29,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 18:53:30,030 [I] Table x_portal_user does not exist in database ranger
2026-08-09 18:53:30,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 18:53:30,273 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 18:53:30,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 18:53:30,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 18:53:30,756 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 18:53:30,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 18:53:31.569 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.584 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.597 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.610 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.623 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.637 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.684 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.690 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.701 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.710 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.719 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.730 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.738 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.744 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.748 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:31.753 UTC [11010] WARNING: there is no transaction in progress
2026-08-09 18:53:32,238 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 18:53:32,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 18:53:32,484 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 18:53:32,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 18:53:32,717 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 18:53:32,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 18:53:32,955 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 18:53:32,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 18:53:33,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 18:53:33,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6zbq.c.gcp-upstream.internal';"
2026-08-09 18:53:33,664 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 18:53:33,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 18:53:33,911 [I] DB_PATCHES have already been applied
I20260809 18:53:33.917164 24427 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-admin:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskUXKsiJ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 18:53:33.917227 24427 mini_ranger.cc:193] Using host: 127.23.218.212
I20260809 18:53:33.920773 24427 mini_ranger.cc:227] Ranger admin URL: http://127.23.218.212:51533
Aug 09, 2026 6:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 6:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 6:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 6:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 6:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 6:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 6:53:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 18:53:35.115852 24427 mini_ranger.cc:160] Time spent starting Ranger: real 7.910s user 0.003s sys 0.005s
Aug 09, 2026 6:53:53 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 6:53:53 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 6:53:53 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 6:53:53 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 6:53:54 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 18:53:55.088279 24427 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.218.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.218.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/rangerkms_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.218.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/rangerkms_127.23.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.218.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.218.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.218.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/HTTP_127.23.218.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.218.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/HTTP_127.23.218.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 18:53:55 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301635, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 18:53:55.162842 24427 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 18:53:55.171269 24427 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 18:53:55.822394 24427 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 18:53:55.822474 24427 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-kms
I20260809 18:53:55.822489 24427 mini_ranger_kms.cc:227] Using postgres at 127.23.218.212:48259
2026-08-09 18:53:55,977 [I] DB FLAVOR :POSTGRES
2026-08-09 18:53:55,978 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 18:53:55,978 [I] Checking connection
2026-08-09 18:53:55,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 18:53:56,212 [I] connection success
2026-08-09 18:53:56,212 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 18:53:56,212 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 18:53:56,212 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 18:53:56,435 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 18:53:56,436 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 18:53:56,436 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 18:53:56,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.218.212:48259/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 18:53:56,675 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 18:53:57.135298 24427 mini_ranger_kms.cc:310] Created kms service
I20260809 18:53:57.353113 24427 mini_ranger_kms.cc:326] Created kudu user
I20260809 18:53:57.393702 24427 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 18:53:57 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301635, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:53:57.768710 24427 mini_ranger_kms.cc:384] Added ranger policy
I20260809 18:53:57.768899 24427 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-kms:/tmp/dist-test-taskUXKsiJ/build/release/bin/postgresql.jar:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskUXKsiJ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 18:53:57.768916 24427 mini_ranger_kms.cc:240] Using host: 127.23.218.212
I20260809 18:53:57.774292 24427 mini_ranger_kms.cc:276] Ranger KMS PID: 11294
I20260809 18:53:57.774394 24427 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.218.212:35561
Aug 09, 2026 6:53:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 6:53:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 6:53:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 6:53:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 6:53:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 6:53:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 18:53:59 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786301639, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.218.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:00.221510 24427 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.059s user 0.006s sys 0.134s
I20260809 18:54:00.221594 24427 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 18:54:00.221642 24427 mini_ranger_kms.cc:401] 127.23.218.212:35561/kms/v1/keys
Aug 09 18:54:00 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301639, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.218.212@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
Aug 09 18:54:00 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301639, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.218.212@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
Aug 09 18:54:02 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:54:02.886677 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34089
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:34089
--ranger_config_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:35561/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:02.965610 11388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:02.965799 11388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:02.965821 11388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:02.966976 11388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:02.967016 11388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:02.967028 11388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:02.967039 11388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:02.967051 11388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:02.968565 11388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.218.212:35561/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:34089
--ranger_config_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:02.968781 11388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:02.968981 11388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:02.973562 11398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:02.973575 11397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:02.973572 11403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:02.973922 11388 server_base.cc:1034] running on GCE node
I20260809 18:54:02.974143 11388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:02.974455 11388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:02.975601 11388 hybrid_clock.cc:648] HybridClock initialized: now 1786301642975585 us; error 33 us; skew 500 ppm
Aug 09 18:54:02 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301642, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:02.978078 11388 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:54:02.978588 11388 webserver.cc:460] Webserver started at http://127.23.218.254:42943/ using document root <none> and password file <none>
I20260809 18:54:02.978822 11388 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:02.978871 11388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:02.978994 11388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 18:54:02 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301642, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:03.031569 11388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "5d3e3cfef13b4df28956b268650f240e"
format_stamp: "Formatted at 2026-08-09 18:54:03 on dist-test-slave-6zbq"
server_key: "e9d4a17fd70165fcb728215844e95d13"
server_key_iv: "110bb2b7a353d83064a9f2875bfcfbf0"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:03.032003 11388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "5d3e3cfef13b4df28956b268650f240e"
format_stamp: "Formatted at 2026-08-09 18:54:03 on dist-test-slave-6zbq"
server_key: "e9d4a17fd70165fcb728215844e95d13"
server_key_iv: "110bb2b7a353d83064a9f2875bfcfbf0"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:03.033712 11388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:54:03.046713 11417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:03.046898 11388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 18:54:03.046983 11388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "5d3e3cfef13b4df28956b268650f240e"
format_stamp: "Formatted at 2026-08-09 18:54:03 on dist-test-slave-6zbq"
server_key: "e9d4a17fd70165fcb728215844e95d13"
server_key_iv: "110bb2b7a353d83064a9f2875bfcfbf0"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:03.047055 11388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:03.072571 11388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:03.074285 11388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:03.074445 11388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:03.078905 11388 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:34089
I20260809 18:54:03.078972 11493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:34089 every 8 connection(s)
I20260809 18:54:03.079291 11388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:54:03.080014 11494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:03.081528 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 11388
I20260809 18:54:03.081621 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:54:03.081804 24427 external_mini_cluster.cc:1428] Setting key c3fe8b55fd2b4fd69d020b726ec37739
I20260809 18:54:03.082986 11494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Bootstrap starting.
I20260809 18:54:03.083679 11494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:03.083971 11494 log.cc:824] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:03.084728 11494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: No bootstrap required, opened a new log
I20260809 18:54:03.087283 11494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } }
I20260809 18:54:03.087448 11494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:03.087479 11494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d3e3cfef13b4df28956b268650f240e, State: Initialized, Role: FOLLOWER
I20260809 18:54:03.087636 11494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } }
Aug 09 18:54:03 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:03.087713 11494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:54:03.087740 11494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:54:03.087803 11494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:03.088474 11494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } }
I20260809 18:54:03.088578 11494 leader_election.cc:302] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d3e3cfef13b4df28956b268650f240e; no voters:
I20260809 18:54:03.088775 11494 leader_election.cc:288] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:54:03.088825 11501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:03.089131 11494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:03.089413 11501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 1 LEADER]: Becoming Leader. State: Replica: 5d3e3cfef13b4df28956b268650f240e, State: Running, Role: LEADER
I20260809 18:54:03.089562 11501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } }
I20260809 18:54:03.089942 11501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d3e3cfef13b4df28956b268650f240e. Latest consensus state: current_term: 1 leader_uuid: "5d3e3cfef13b4df28956b268650f240e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } } }
I20260809 18:54:03.090600 11501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:03.091084 11502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d3e3cfef13b4df28956b268650f240e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } } }
I20260809 18:54:03.091187 11502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:03.091786 11494 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 18:54:03.091913 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:03.082238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50102 (local address 127.23.218.254:34089)
0809 18:54:03.082414 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:03.082423 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:03.082498 (+ 75us) server_negotiation.cc:407] Connection header received
0809 18:54:03.082930 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:03.082937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:03.082983 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:03.083103 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:03.083723 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:03.084748 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:03.085480 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:03.085880 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:03.085974 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:03.088242 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:03.088246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:03.088246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:03.088256 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:03.089928 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:03.090287 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:03.090290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:03.090290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:03.090406 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:03.090588 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:03.090588 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:03.090589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:03.090852 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:03.091498 (+ 646us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:03.091569 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 18:54:03.091677 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":69,"threads_started":1}
I20260809 18:54:03.092034 11522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:54:03.092494 11522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:54:03.093777 11522 catalog_manager.cc:1347] Generated new cluster ID: 6e29adf6f17141b987230b1c963ac70c
I20260809 18:54:03.093827 11522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:54:03.128539 11522 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:54:03.129319 11522 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:54:03.141353 11522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Generated new TSK 0
I20260809 18:54:03.141590 11522 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:54:03.970594 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:35561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34089
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:04.055667 11562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:04.055855 11562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:04.055876 11562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:04.056999 11562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:04.057039 11562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:04.057063 11562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:54:04.058511 11562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.218.212:35561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:04.058704 11562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:04.058910 11562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:04.064522 11578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:04.064682 11581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:04.064544 11576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:04.064821 11562 server_base.cc:1034] running on GCE node
I20260809 18:54:04.065001 11562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:04.065212 11562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:04.066367 11562 hybrid_clock.cc:648] HybridClock initialized: now 1786301644066352 us; error 30 us; skew 500 ppm
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:04.068693 11562 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:54:04.069064 11562 webserver.cc:460] Webserver started at http://127.23.218.193:38019/ using document root <none> and password file <none>
I20260809 18:54:04.069274 11562 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:04.069324 11562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:04.069434 11562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:04.083648 11562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "c4cc8bbd241540b4b93eb345cf1d4aed"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "92774618ccf509442a852e238d4059ca"
server_key_iv: "44ccb28b397764c169d503ba30cdfd07"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:04.084050 11562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "c4cc8bbd241540b4b93eb345cf1d4aed"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "92774618ccf509442a852e238d4059ca"
server_key_iv: "44ccb28b397764c169d503ba30cdfd07"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:04.085827 11562 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:04.098575 11594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.099939 11562 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 18:54:04.100057 11562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "c4cc8bbd241540b4b93eb345cf1d4aed"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "92774618ccf509442a852e238d4059ca"
server_key_iv: "44ccb28b397764c169d503ba30cdfd07"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:04.100154 11562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:04.117466 11562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:04.119647 11562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:04.119818 11562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:04.120075 11562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:04.120450 11562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:04.120505 11562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.120539 11562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:04.120594 11562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.127916 11562 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:46631
I20260809 18:54:04.128010 11786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:46631 every 8 connection(s)
I20260809 18:54:04.128314 11562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:54:04.135449 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 11562
I20260809 18:54:04.135556 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:54:04.135751 24427 external_mini_cluster.cc:1428] Setting key b85d6c32e6df236e00af0409a76a73e0
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:04.139286 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:04.129076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:58571 (local address 127.23.218.254:34089)
0809 18:54:04.129290 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:04.129291 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:04.129344 (+ 53us) server_negotiation.cc:407] Connection header received
0809 18:54:04.130013 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:04.130014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:04.130026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:04.130080 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:04.130819 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.131606 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.132842 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.133146 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.133225 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:04.136211 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:04.136213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:04.136213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:04.136233 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:04.137847 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:04.138313 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:04.138315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:04.138315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:04.138404 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:04.138640 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:04.138641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:04.138642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:04.138782 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:04.138840 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:04.139093 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 18:54:04.139184 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":72,"threads_started":1}
I20260809 18:54:04.139415 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:04.129028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34089 (local address 127.23.218.193:58571)
0809 18:54:04.129222 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:04.129269 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:04.129911 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:04.130153 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:04.130157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:04.130423 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:04.130698 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:04.130716 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.131730 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.131731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:04.132746 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:04.132750 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.133273 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.133274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:04.133414 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:04.134589 (+ 1175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:04.134600 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:04.136098 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:04.137936 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:04.137945 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:04.137948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:04.138242 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:04.138468 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:04.138468 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:04.138469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:04.138574 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:04.138839 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:04.138846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:04.138975 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 18:54:04.139062 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 18:54:04.139199 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260809 18:54:04.139886 11788 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34089
I20260809 18:54:04.139983 11788 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:04.140161 11788 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
I20260809 18:54:04.140733 11440 ts_manager.cc:194] Registered new tserver with Master: c4cc8bbd241540b4b93eb345cf1d4aed (127.23.218.193:46631)
I20260809 18:54:04.141654 11440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:58571
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:54:04.162698 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:35561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34089
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:04.271883 11799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:04.272059 11799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:04.272089 11799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:04.273180 11799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:04.273227 11799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:04.273262 11799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:54:04.274744 11799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.218.212:35561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:04.275035 11799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:04.275251 11799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:04.279850 11814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:04.280025 11812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:04.279902 11818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:04.280812 11799 server_base.cc:1034] running on GCE node
I20260809 18:54:04.280987 11799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:04.281215 11799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:04.282387 11799 hybrid_clock.cc:648] HybridClock initialized: now 1786301644282369 us; error 35 us; skew 500 ppm
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:04.284686 11799 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:54:04.285045 11799 webserver.cc:460] Webserver started at http://127.23.218.194:40903/ using document root <none> and password file <none>
I20260809 18:54:04.285255 11799 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:04.285302 11799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:04.285413 11799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:04.297878 11799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "f4fee278fe954fe69f143dbf9e1f4060"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "1fa5ef33aa2df666c1dee3d2793c0fbb"
server_key_iv: "2c0324c1b47534688de42c451bfc49a9"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:04.298377 11799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "f4fee278fe954fe69f143dbf9e1f4060"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "1fa5ef33aa2df666c1dee3d2793c0fbb"
server_key_iv: "2c0324c1b47534688de42c451bfc49a9"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:04.304831 11799 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.001s
I20260809 18:54:04.321485 11830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.322878 11799 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260809 18:54:04.322975 11799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "f4fee278fe954fe69f143dbf9e1f4060"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "1fa5ef33aa2df666c1dee3d2793c0fbb"
server_key_iv: "2c0324c1b47534688de42c451bfc49a9"
server_key_version: "kuduclusterkey@0"
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:04.323047 11799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:04.337154 11799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:04.339398 11799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:04.339591 11799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:04.339871 11799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:04.340255 11799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:04.340342 11799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.340411 11799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:04.340466 11799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.348940 11799 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:43167
I20260809 18:54:04.349395 11799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:54:04.349951 12006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:43167 every 8 connection(s)
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:04.358210 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 11799
I20260809 18:54:04.358312 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:54:04.358506 24427 external_mini_cluster.cc:1428] Setting key 358fc5198007dc4cebf4c9f853162591
I20260809 18:54:04.361925 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:04.350851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:44965 (local address 127.23.218.254:34089)
0809 18:54:04.350904 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:04.350906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:04.351348 (+ 442us) server_negotiation.cc:407] Connection header received
0809 18:54:04.352141 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:04.352142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:04.352157 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:04.352187 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:04.352977 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.353764 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.355277 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.355593 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.355641 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:04.358189 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:04.358191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:04.358191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:04.358199 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:04.360366 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:04.360831 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:04.360833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:04.360833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:04.360897 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:04.361189 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:04.361190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:04.361190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:04.361329 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:04.361376 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:04.361747 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 18:54:04.361840 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 18:54:04.362625 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:04.350576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34089 (local address 127.23.218.194:44965)
0809 18:54:04.351226 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:04.351267 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:04.352019 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:04.352366 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:04.352370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:04.352605 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:04.352865 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:04.352872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.353885 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.353886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:04.355172 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:04.355176 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.355766 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.355767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:04.355904 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:04.356404 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:04.356414 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:04.357997 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:04.360463 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:04.360470 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:04.360472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:04.360742 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:04.360968 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:04.360968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:04.360969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:04.361109 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:04.361425 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:04.361431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:04.361578 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 18:54:04.362221 (+ 643us) client_negotiation.cc:240] Negotiation successful
0809 18:54:04.362402 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":86,"threads_started":1}
I20260809 18:54:04.363216 12007 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34089
I20260809 18:54:04.363301 12007 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:04.363464 12007 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
I20260809 18:54:04.363884 11440 ts_manager.cc:194] Registered new tserver with Master: f4fee278fe954fe69f143dbf9e1f4060 (127.23.218.194:43167)
I20260809 18:54:04.364341 11440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:44965
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:54:04.398207 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:35561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34089
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:04.505501 12016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:04.505820 12016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:04.505903 12016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:04.507579 12016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:04.507638 12016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:04.507675 12016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:54:04.509759 12016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.218.212:35561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:04.509994 12016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:04.510277 12016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:04.518414 12026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:04.518630 12024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:04.518822 12023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:04.519860 12016 server_base.cc:1034] running on GCE node
I20260809 18:54:04.520128 12016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:04.520399 12016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:04.523691 12016 hybrid_clock.cc:648] HybridClock initialized: now 1786301644523665 us; error 35 us; skew 500 ppm
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:04.526831 12016 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:54:04.527329 12016 webserver.cc:460] Webserver started at http://127.23.218.195:36631/ using document root <none> and password file <none>
I20260809 18:54:04.527599 12016 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:04.527673 12016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:04.527812 12016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:04.548234 12016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "2628f27f356a4e848a8eb3cffba50c0f"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "8deac8efdeba54a9eb94a822584152d8"
server_key_iv: "21e218f8f495309fffa48ec38b388919"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:04.548647 12016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "2628f27f356a4e848a8eb3cffba50c0f"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "8deac8efdeba54a9eb94a822584152d8"
server_key_iv: "21e218f8f495309fffa48ec38b388919"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:04.552408 12016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 18:54:04.573540 12036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.573704 12016 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260809 18:54:04.573789 12016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "2628f27f356a4e848a8eb3cffba50c0f"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "8deac8efdeba54a9eb94a822584152d8"
server_key_iv: "21e218f8f495309fffa48ec38b388919"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:04.573868 12016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:04.618515 12016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:04.632278 12016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:04.632429 12016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:04.632668 12016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:04.633054 12016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:04.633103 12016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.633138 12016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:04.633159 12016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.646555 12016 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:40661
I20260809 18:54:04.647009 12016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:54:04.653514 12205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:40661 every 8 connection(s)
I20260809 18:54:04.654724 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 12016
I20260809 18:54:04.654836 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:54:04.654999 24427 external_mini_cluster.cc:1428] Setting key a7c0e2c5f4907e83c1be8208726b78f2
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:04.658631 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:04.648378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:46661 (local address 127.23.218.254:34089)
0809 18:54:04.648423 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:04.648425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:04.648661 (+ 236us) server_negotiation.cc:407] Connection header received
0809 18:54:04.649542 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:04.649544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:04.649556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:04.649585 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:04.650570 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.651246 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.652632 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.652888 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.652925 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:04.655535 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:04.655536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:04.655536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:04.655545 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:04.656945 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:04.657404 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:04.657405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:04.657405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:04.657477 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:04.657817 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:04.657819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:04.657819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:04.657965 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:04.658015 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:04.658429 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 18:54:04.658527 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:54:04.659168 12209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:04.648056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34089 (local address 127.23.218.195:46661)
0809 18:54:04.648534 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:04.648575 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:04.649416 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:04.649760 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:04.649764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:04.650120 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:04.650437 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:04.650476 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.651368 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.651369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:04.652535 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:04.652539 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:04.653044 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:04.653045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:04.653188 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:04.653864 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:04.653874 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:04.655433 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:04.657026 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:04.657032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:04.657035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:04.657311 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:04.657564 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:04.657565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:04.657565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:04.657721 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:04.658068 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:04.658074 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:04.658236 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 18:54:04.658773 (+ 537us) client_negotiation.cc:240] Negotiation successful
0809 18:54:04.658941 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":128,"threads_started":1}
I20260809 18:54:04.659657 12206 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34089
I20260809 18:54:04.659755 12206 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:04.659943 12206 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
I20260809 18:54:04.660292 11440 ts_manager.cc:194] Registered new tserver with Master: 2628f27f356a4e848a8eb3cffba50c0f (127.23.218.195:40661)
I20260809 18:54:04.660758 11440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:46661
I20260809 18:54:04.666554 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:54:04.667408 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 11562
I20260809 18:54:04.694556 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 11799
I20260809 18:54:04.700625 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 12016
I20260809 18:54:04.711385 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 11388
I20260809 18:54:04.726238 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34089
--webserver_interface=127.23.218.254
--webserver_port=42943
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:34089
--ranger_config_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:35561/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.218.212:35561/kms with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:04.840417 12210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:04.840687 12210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:04.840720 12210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:04.842399 12210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:04.842449 12210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:04.842468 12210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:04.842484 12210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:04.842507 12210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:04.844666 12210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.218.212:35561/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:34089
--ranger_config_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=42943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:04.844892 12210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:04.845106 12210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:04.849776 12216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:04.849783 12225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:04.849818 12210 server_base.cc:1034] running on GCE node
W20260809 18:54:04.849785 12228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:04.850158 12210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:04.850454 12210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:04.851606 12210 hybrid_clock.cc:648] HybridClock initialized: now 1786301644851592 us; error 28 us; skew 500 ppm
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:04.853782 12210 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:54:04.854187 12210 webserver.cc:460] Webserver started at http://127.23.218.254:42943/ using document root <none> and password file <none>
I20260809 18:54:04.854399 12210 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:04.854455 12210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 18:54:04 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301644, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:04.927347 12210 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:04.927999 12239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:04.928153 12210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:04.928244 12210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "5d3e3cfef13b4df28956b268650f240e"
format_stamp: "Formatted at 2026-08-09 18:54:03 on dist-test-slave-6zbq"
server_key: "e9d4a17fd70165fcb728215844e95d13"
server_key_iv: "110bb2b7a353d83064a9f2875bfcfbf0"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:04.928599 12210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:04.941854 12210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:04.943301 12210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:04.943429 12210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:04.947269 12210 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:34089
I20260809 18:54:04.947329 12324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:34089 every 8 connection(s)
I20260809 18:54:04.947649 12210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:54:04.948786 12326 sys_catalog.cc:263] Verifying existing consensus state
I20260809 18:54:04.949415 12326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Bootstrap starting.
I20260809 18:54:04.952189 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 12210
I20260809 18:54:04.952504 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:35561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:46631
--local_ip_for_outbound_sockets=127.23.218.193
--tserver_master_addrs=127.23.218.254:34089
--webserver_port=38019
--webserver_interface=127.23.218.193
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.218.212:35561/kms with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:54:04.957732 12326 log.cc:824] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:04.959260 12326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:54:04.959487 12326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Bootstrap complete.
I20260809 18:54:04.961730 12326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } }
I20260809 18:54:04.961954 12326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d3e3cfef13b4df28956b268650f240e, State: Initialized, Role: FOLLOWER
I20260809 18:54:04.962105 12326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } }
I20260809 18:54:04.962180 12326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:54:04.962226 12326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:54:04.962272 12326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 1 FOLLOWER]: Advancing to term 2
I20260809 18:54:04.963204 12326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } }
I20260809 18:54:04.963299 12326 leader_election.cc:302] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d3e3cfef13b4df28956b268650f240e; no voters:
I20260809 18:54:04.963444 12326 leader_election.cc:288] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 18:54:04.963500 12331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 18:54:04.963673 12326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:04.963673 12331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [term 2 LEADER]: Becoming Leader. State: Replica: 5d3e3cfef13b4df28956b268650f240e, State: Running, Role: LEADER
I20260809 18:54:04.963965 12331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } }
I20260809 18:54:04.964234 12333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5d3e3cfef13b4df28956b268650f240e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } } }
I20260809 18:54:04.964295 12333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:04.964814 12332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d3e3cfef13b4df28956b268650f240e. Latest consensus state: current_term: 2 leader_uuid: "5d3e3cfef13b4df28956b268650f240e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d3e3cfef13b4df28956b268650f240e" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34089 } } }
I20260809 18:54:04.964884 12332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:04.965018 12345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:54:04.965183 12326 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 18:54:04.965469 12345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:54:04.967406 12345 catalog_manager.cc:1259] Loaded cluster ID: 6e29adf6f17141b987230b1c963ac70c
I20260809 18:54:04.967478 12345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:54:04.971768 12345 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:54:04.972012 12345 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Loaded TSK: 0
I20260809 18:54:04.972301 12345 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 18:54:05.037635 12329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:05.037917 12329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:05.037969 12329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:05.039588 12329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:05.039649 12329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:05.039686 12329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:54:05.041787 12329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.218.212:35561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:46631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=38019
--tserver_master_addrs=127.23.218.254:34089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:05.042153 12329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:05.042462 12329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:05.048799 12401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:05.048768 12397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:05.049034 12396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:05.049162 12329 server_base.cc:1034] running on GCE node
I20260809 18:54:05.049373 12329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:05.049630 12329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:05.050763 12329 hybrid_clock.cc:648] HybridClock initialized: now 1786301645050737 us; error 42 us; skew 500 ppm
Aug 09 18:54:05 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301645, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:05.054667 12329 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:54:05.055154 12329 webserver.cc:460] Webserver started at http://127.23.218.193:38019/ using document root <none> and password file <none>
I20260809 18:54:05.055428 12329 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:05.055517 12329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 18:54:05 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301645, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:05.078688 12329 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260809 18:54:05.079906 12415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:05.080068 12329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:05.080160 12329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "c4cc8bbd241540b4b93eb345cf1d4aed"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "92774618ccf509442a852e238d4059ca"
server_key_iv: "44ccb28b397764c169d503ba30cdfd07"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:05.080626 12329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:05.122278 12329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:05.124595 12329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:05.124775 12329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:05.125005 12329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:05.125362 12329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:05.125432 12329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:05.125484 12329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:05.125527 12329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:05.133062 12329 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:46631
I20260809 18:54:05.133131 12593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:46631 every 8 connection(s)
I20260809 18:54:05.133553 12329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:54:05.138756 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 12329
I20260809 18:54:05.139048 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:35561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:43167
--local_ip_for_outbound_sockets=127.23.218.194
--tserver_master_addrs=127.23.218.254:34089
--webserver_port=40903
--webserver_interface=127.23.218.194
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.218.212:35561/kms with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 18:54:05 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301645, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:05.146751 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:05.134290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34089 (local address 127.23.218.193:47255)
0809 18:54:05.134626 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:05.134666 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:05.135304 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:05.135935 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:05.135939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:05.136136 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:05.136395 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:05.136401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.137833 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.137833 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:05.139043 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:05.139046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.139932 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.139933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:05.140069 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:05.140564 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:05.140574 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:05.142088 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:05.144993 (+ 2905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:05.145000 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:05.145002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:05.145280 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:05.145543 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:05.145544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:05.145544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:05.145673 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:05.146116 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:05.146122 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:05.146260 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 18:54:05.146377 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 18:54:05.146511 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":112,"threads_started":1}
I20260809 18:54:05.147454 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:05.134367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:47255 (local address 127.23.218.254:34089)
0809 18:54:05.134546 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:05.134554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:05.134751 (+ 197us) server_negotiation.cc:407] Connection header received
0809 18:54:05.135556 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:05.135564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:05.135619 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:05.135723 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:05.136521 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.137710 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.139141 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.139713 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.139794 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:05.142210 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:05.142215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:05.142215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:05.142238 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:05.144912 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:05.145377 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:05.145380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:05.145380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:05.145478 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:05.145749 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:05.145750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:05.145750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:05.146017 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:05.146089 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:05.147151 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0809 18:54:05.147266 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260809 18:54:05.148593 12594 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34089
I20260809 18:54:05.148712 12594 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:05.148916 12594 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
W20260809 18:54:05.149863 12594 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:34089 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 18:54:05.149940 12594 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:05.150010 12594 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
I20260809 18:54:05.150398 12594 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:05.150475 12594 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
W20260809 18:54:05.150846 12594 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 18:54:05.246316 12600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:05.246593 12600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:05.246630 12600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:05.248157 12600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:05.248260 12600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:05.248337 12600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:54:05.250144 12600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.218.212:35561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:43167
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=40903
--tserver_master_addrs=127.23.218.254:34089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:05.250536 12600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:05.250818 12600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:05.256896 12607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:05.257242 12609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:05.257489 12606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:05.258378 12600 server_base.cc:1034] running on GCE node
I20260809 18:54:05.258577 12600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:05.258863 12600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:05.262151 12600 hybrid_clock.cc:648] HybridClock initialized: now 1786301645262126 us; error 35 us; skew 500 ppm
Aug 09 18:54:05 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301645, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:05.265249 12600 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:54:05.265749 12600 webserver.cc:460] Webserver started at http://127.23.218.194:40903/ using document root <none> and password file <none>
I20260809 18:54:05.265986 12600 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:05.266059 12600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 18:54:05 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301645, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:05.290530 12600 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260809 18:54:05.291539 12620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:05.291764 12600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 18:54:05.291864 12600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "f4fee278fe954fe69f143dbf9e1f4060"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "1fa5ef33aa2df666c1dee3d2793c0fbb"
server_key_iv: "2c0324c1b47534688de42c451bfc49a9"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:05.292305 12600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:05.304278 12600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:05.306391 12600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:05.306546 12600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:05.306810 12600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:05.307168 12600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:05.307219 12600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:05.307250 12600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:05.307273 12600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:05.314266 12600 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:43167
I20260809 18:54:05.314368 12797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:43167 every 8 connection(s)
I20260809 18:54:05.314677 12600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:54:05.319125 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 12600
I20260809 18:54:05.319401 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.218.212:35561/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:40661
--local_ip_for_outbound_sockets=127.23.218.195
--tserver_master_addrs=127.23.218.254:34089
--webserver_port=36631
--webserver_interface=127.23.218.195
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.218.212:35561/kms with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 18:54:05 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301645, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:05.325346 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:05.315375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34089 (local address 127.23.218.194:50347)
0809 18:54:05.315532 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:05.315573 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:05.316036 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:05.316275 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:05.316280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:05.316445 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:05.316694 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:05.316702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.317560 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.317561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:05.318363 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:05.318367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.318799 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.318799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:05.318892 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:05.319427 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:05.319437 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:05.321919 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:05.323671 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:05.323680 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:05.323682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:05.323991 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:05.324245 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:05.324246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:05.324247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:05.324388 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:05.324687 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:05.324693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:05.324866 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 18:54:05.324972 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 18:54:05.325123 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260809 18:54:05.325809 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:05.315371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:50347 (local address 127.23.218.254:34089)
0809 18:54:05.315417 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:05.315419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:05.315646 (+ 227us) server_negotiation.cc:407] Connection header received
0809 18:54:05.316135 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:05.316137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:05.316150 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:05.316201 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:05.316819 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.317451 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.318467 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.318680 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.318714 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:05.322008 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:05.322010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:05.322010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:05.322018 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:05.323573 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:05.324072 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:05.324073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:05.324073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:05.324154 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:05.324463 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:05.324464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:05.324465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:05.324599 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:05.324660 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:05.325618 (+ 958us) server_negotiation.cc:299] Negotiation successful
0809 18:54:05.325720 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:54:05.326126 12798 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34089
I20260809 18:54:05.326226 12798 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:05.326423 12798 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
W20260809 18:54:05.327059 12798 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:34089 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 18:54:05.327118 12798 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:05.327188 12798 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
I20260809 18:54:05.327565 12798 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:05.327670 12798 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
W20260809 18:54:05.328007 12798 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 18:54:05.440470 12802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:05.440738 12802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:05.440774 12802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:05.442447 12802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:05.442499 12802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:05.442534 12802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:54:05.444651 12802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.218.212:35561/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:40661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=36631
--tserver_master_addrs=127.23.218.254:34089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:05.444883 12802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:05.445140 12802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:05.452049 12817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:05.452333 12802 server_base.cc:1034] running on GCE node
W20260809 18:54:05.452265 12813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:05.453830 12815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:05.454061 12802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:05.454362 12802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:05.455538 12802 hybrid_clock.cc:648] HybridClock initialized: now 1786301645455508 us; error 38 us; skew 500 ppm
Aug 09 18:54:05 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301645, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:05.458933 12802 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:54:05.459477 12802 webserver.cc:460] Webserver started at http://127.23.218.195:36631/ using document root <none> and password file <none>
I20260809 18:54:05.459762 12802 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:05.459851 12802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 18:54:05 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301645, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:05.479211 12802 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260809 18:54:05.481791 12827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:05.481945 12802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260809 18:54:05.482014 12802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "2628f27f356a4e848a8eb3cffba50c0f"
format_stamp: "Formatted at 2026-08-09 18:54:04 on dist-test-slave-6zbq"
server_key: "8deac8efdeba54a9eb94a822584152d8"
server_key_iv: "21e218f8f495309fffa48ec38b388919"
server_key_version: "kuduclusterkey@0"
I20260809 18:54:05.482522 12802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:05.498911 12802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:05.500720 12802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:05.500865 12802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:05.501132 12802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:05.501473 12802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:05.501513 12802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:05.501540 12802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:05.501561 12802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:05.511219 12802 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:40661
I20260809 18:54:05.511854 12802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:54:05.512633 13001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:40661 every 8 connection(s)
I20260809 18:54:05.517073 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 12802
Aug 09 18:54:05 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301645, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:05.523445 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:05.512877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34089 (local address 127.23.218.195:50323)
0809 18:54:05.513110 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:05.513145 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:05.513723 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:05.514008 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:05.514012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:05.514204 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:05.514464 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:05.514469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.515390 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.515391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:05.516175 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:05.516178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.516688 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.516689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:05.516792 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:05.518510 (+ 1718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:05.518520 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:05.520099 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:05.521922 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:05.521930 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:05.521932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:05.522140 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:05.522336 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:05.522358 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:05.522358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:05.522486 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:05.522782 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:05.522789 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:05.522944 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 18:54:05.523050 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 18:54:05.523201 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":112,"threads_started":1}
I20260809 18:54:05.523910 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:05.512982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:50323 (local address 127.23.218.254:34089)
0809 18:54:05.513036 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:05.513038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:05.513230 (+ 192us) server_negotiation.cc:407] Connection header received
0809 18:54:05.513849 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:05.513852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:05.513866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:05.513936 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:05.514557 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.515285 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.516263 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.516544 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.516627 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:05.520198 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:05.520200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:05.520200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:05.520208 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:05.521828 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:05.522220 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:05.522222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:05.522222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:05.522280 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:05.522561 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:05.522562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:05.522562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:05.522699 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:05.522771 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:05.523743 (+ 972us) server_negotiation.cc:299] Negotiation successful
0809 18:54:05.523819 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:54:05.524291 13004 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34089
I20260809 18:54:05.524380 13004 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:05.524557 13004 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
W20260809 18:54:05.525434 13004 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:34089 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 18:54:05.525728 13004 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:05.525801 13004 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
I20260809 18:54:05.526113 13004 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:05.526187 13004 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
W20260809 18:54:05.526506 13004 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 18:54:05.527604 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:05.517476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.23.218.254:34089)
0809 18:54:05.519187 (+ 1711us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:05.519188 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:05.519195 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:54:05.519212 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:05.519212 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:05.519221 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:05.519292 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:05.520522 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.521368 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.522467 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:05.522830 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:05.522885 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:05.524539 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:05.524540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:05.524541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:05.524547 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:05.526509 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:05.526846 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:05.526846 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:05.526847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:05.526890 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:05.527072 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:05.527072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:05.527073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:05.527174 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:05.527320 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:05.527358 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 18:54:05.527433 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1693,"thread_start_us":56,"threads_started":1}
Aug 09 18:54:06 dist-test-slave-6zbq krb5kdc[10694](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786301646, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:06.151017 12594 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:06.151167 12594 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
I20260809 18:54:06.151567 12267 ts_manager.cc:194] Registered new tserver with Master: c4cc8bbd241540b4b93eb345cf1d4aed (127.23.218.193:46631)
I20260809 18:54:06.152530 12267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:47255
I20260809 18:54:06.328190 12798 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:06.328329 12798 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
I20260809 18:54:06.328594 12267 ts_manager.cc:194] Registered new tserver with Master: f4fee278fe954fe69f143dbf9e1f4060 (127.23.218.194:43167)
I20260809 18:54:06.328970 12267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:50347
Aug 09 18:54:06 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301646, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:06.526669 13004 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:06.526813 13004 heartbeater.cc:511] Master 127.23.218.254:34089 requested a full tablet report, sending...
I20260809 18:54:06.527096 12267 ts_manager.cc:194] Registered new tserver with Master: 2628f27f356a4e848a8eb3cffba50c0f (127.23.218.195:40661)
I20260809 18:54:06.527616 12267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:50323
I20260809 18:54:06.529469 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:54:06.549618 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.530393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57204 (local address 127.23.218.254:34089)
0809 18:54:06.542412 (+ 12019us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:06.542414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:06.542421 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:54:06.542441 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:06.542441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:06.542451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:06.542524 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:06.543232 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.543994 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.545015 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.545300 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.545361 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.546789 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:06.546790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:06.546791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:06.546798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:06.548452 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:06.548831 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:06.548832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:06.548833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:06.548893 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:06.549085 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:06.549085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:06.549085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:06.549210 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:06.549304 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:06.549439 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 18:54:06.549521 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11993,"thread_start_us":81,"threads_started":1}
I20260809 18:54:06.551662 12267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:54:06.553769 12267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:54:06.563378 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.556572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:43167 (local address 127.0.0.1:40368)
0809 18:54:06.557192 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:06.557243 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:06.557313 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:06.559308 (+ 1995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:06.559309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:06.559311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:06.559389 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.559391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.560764 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.560765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.562083 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.562086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.563077 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.563078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.563171 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.563188 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 18:54:06.563250 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 18:54:06.563303 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":27,"thread_start_us":84,"threads_started":1}
I20260809 18:54:06.563884 13042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.556664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40368 (local address 127.23.218.194:43167)
0809 18:54:06.559076 (+ 2412us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:06.559080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:06.559090 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:54:06.559148 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:06.559151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:06.559167 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:06.559528 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:06.559651 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.560656 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.562180 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.562971 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.563668 (+ 697us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.563739 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:06.563766 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 18:54:06.563807 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2349,"thread_start_us":64,"threads_started":1}
I20260809 18:54:06.564677 12697 tablet_service.cc:1467] Processing CreateTablet for tablet 18cbaa995cff426391aa42625199143a (DEFAULT_TABLE table=test-table [id=4a2db8f3be034d7ca2be1c4cb4efeb27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:06.565065 12697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18cbaa995cff426391aa42625199143a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:06.567778 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.556949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46631 (local address 127.0.0.1:58324)
0809 18:54:06.558454 (+ 1505us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:06.558464 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:06.558528 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:06.558811 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:06.558815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:06.558817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:06.558873 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.558875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.560713 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.560713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.566182 (+ 5469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.566185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.567590 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.567591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.567640 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.567644 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:54:06.567680 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 18:54:06.567715 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1487,"thread_start_us":52,"threads_started":1}
I20260809 18:54:06.568028 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.557054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.23.218.193:46631)
0809 18:54:06.557447 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:06.557451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:06.558539 (+ 1088us) server_negotiation.cc:407] Connection header received
0809 18:54:06.558603 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:06.558605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:06.558631 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:06.558705 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:06.558955 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.560481 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.566278 (+ 5797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.567033 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.567101 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.567150 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:06.567914 (+ 764us) server_negotiation.cc:299] Negotiation successful
0809 18:54:06.567955 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":73,"threads_started":1}
I20260809 18:54:06.568629 12486 tablet_service.cc:1467] Processing CreateTablet for tablet 18cbaa995cff426391aa42625199143a (DEFAULT_TABLE table=test-table [id=4a2db8f3be034d7ca2be1c4cb4efeb27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:06.568940 12486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18cbaa995cff426391aa42625199143a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:06.570380 13049 tablet_bootstrap.cc:492] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: Bootstrap starting.
I20260809 18:54:06.571002 13049 tablet_bootstrap.cc:654] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:06.571290 13049 log.cc:824] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:06.572026 13049 tablet_bootstrap.cc:492] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: No bootstrap required, opened a new log
I20260809 18:54:06.572085 13049 ts_tablet_manager.cc:1397] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 18:54:06.573947 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.557114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49472 (local address 127.23.218.195:40661)
0809 18:54:06.558664 (+ 1550us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:06.558669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:06.558706 (+ 37us) server_negotiation.cc:407] Connection header received
0809 18:54:06.558740 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:06.558742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:06.558758 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:06.558895 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:06.562465 (+ 3570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.563821 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.572830 (+ 9009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.573521 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.573588 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.573637 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:06.573832 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 18:54:06.573873 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1518,"thread_start_us":72,"threads_started":1}
I20260809 18:54:06.574287 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.556794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:40661 (local address 127.0.0.1:49472)
0809 18:54:06.557492 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:06.557501 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:06.557540 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:06.560601 (+ 3061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:06.560602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:06.560605 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:06.560645 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.560647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.563988 (+ 3341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.563989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.567508 (+ 3519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.567511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.573696 (+ 6185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.573697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.573756 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.573759 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:54:06.574182 (+ 423us) client_negotiation.cc:240] Negotiation successful
0809 18:54:06.574226 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":79,"threads_started":1}
I20260809 18:54:06.574572 13049 raft_consensus.cc:348] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } }
I20260809 18:54:06.574733 13049 raft_consensus.cc:374] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:06.574766 13049 raft_consensus.cc:729] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4fee278fe954fe69f143dbf9e1f4060, State: Initialized, Role: FOLLOWER
I20260809 18:54:06.574908 13049 consensus_queue.cc:260] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } }
I20260809 18:54:06.575156 13049 ts_tablet_manager.cc:1428] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 18:54:06.575168 12904 tablet_service.cc:1467] Processing CreateTablet for tablet 18cbaa995cff426391aa42625199143a (DEFAULT_TABLE table=test-table [id=4a2db8f3be034d7ca2be1c4cb4efeb27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:06.575302 12798 heartbeater.cc:503] Master 127.23.218.254:34089 was elected leader, sending a full tablet report...
I20260809 18:54:06.575484 12904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18cbaa995cff426391aa42625199143a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:06.578452 13052 tablet_bootstrap.cc:492] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed: Bootstrap starting.
I20260809 18:54:06.578831 13055 tablet_bootstrap.cc:492] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: Bootstrap starting.
I20260809 18:54:06.578962 13052 tablet_bootstrap.cc:654] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:06.579285 13052 log.cc:824] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:06.579401 13055 tablet_bootstrap.cc:654] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:06.579725 13055 log.cc:824] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:06.580163 13052 tablet_bootstrap.cc:492] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed: No bootstrap required, opened a new log
I20260809 18:54:06.580235 13052 ts_tablet_manager.cc:1397] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:54:06.580493 13055 tablet_bootstrap.cc:492] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: No bootstrap required, opened a new log
I20260809 18:54:06.580564 13055 ts_tablet_manager.cc:1397] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:54:06.582094 13052 raft_consensus.cc:348] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } }
I20260809 18:54:06.582204 13052 raft_consensus.cc:374] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:06.582222 13052 raft_consensus.cc:729] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4cc8bbd241540b4b93eb345cf1d4aed, State: Initialized, Role: FOLLOWER
I20260809 18:54:06.582306 13052 consensus_queue.cc:260] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } }
I20260809 18:54:06.582543 13052 ts_tablet_manager.cc:1428] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 18:54:06.582639 12594 heartbeater.cc:503] Master 127.23.218.254:34089 was elected leader, sending a full tablet report...
I20260809 18:54:06.582639 13055 raft_consensus.cc:348] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } }
I20260809 18:54:06.582793 13055 raft_consensus.cc:374] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:06.582830 13055 raft_consensus.cc:729] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2628f27f356a4e848a8eb3cffba50c0f, State: Initialized, Role: FOLLOWER
I20260809 18:54:06.582932 13055 consensus_queue.cc:260] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } }
I20260809 18:54:06.583159 13055 ts_tablet_manager.cc:1428] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 18:54:06.583432 13004 heartbeater.cc:503] Master 127.23.218.254:34089 was elected leader, sending a full tablet report...
W20260809 18:54:06.634975 12595 tablet.cc:2369] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:54:06.643339 13053 raft_consensus.cc:482] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:06.643461 13053 raft_consensus.cc:504] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } }
I20260809 18:54:06.644784 13053 leader_election.cc:288] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2628f27f356a4e848a8eb3cffba50c0f (127.23.218.195:40661), c4cc8bbd241540b4b93eb345cf1d4aed (127.23.218.193:46631)
I20260809 18:54:06.649377 13061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.643965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:40661 (local address 127.23.218.194:49309)
0809 18:54:06.644949 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:06.644959 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:06.645041 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:06.645221 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:06.645222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:06.645224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:06.645304 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.645307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.646157 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.646158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.647354 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.647357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.649120 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.649122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.649192 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.649197 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:54:06.649249 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 18:54:06.649288 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"thread_start_us":93,"threads_started":1}
I20260809 18:54:06.649428 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.644711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:52277 (local address 127.23.218.193:46631)
0809 18:54:06.644764 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:06.644766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:06.645794 (+ 1028us) server_negotiation.cc:407] Connection header received
0809 18:54:06.645903 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:06.645905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:06.645916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:06.645972 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:06.646477 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.647082 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.648493 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.649030 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.649076 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.649083 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:06.649320 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 18:54:06.649365 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 18:54:06.649644 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.644517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46631 (local address 127.23.218.194:52277)
0809 18:54:06.645725 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:06.645735 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:06.645820 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:06.646064 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:06.646065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:06.646067 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:06.646130 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.646131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.647181 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.647181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.648393 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:06.648396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.649142 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.649142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:06.649234 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.649239 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:54:06.649527 (+ 288us) client_negotiation.cc:240] Negotiation successful
0809 18:54:06.649568 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1189,"thread_start_us":89,"threads_started":1}
I20260809 18:54:06.649914 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.644176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:49309 (local address 127.23.218.195:40661)
0809 18:54:06.644221 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:06.644223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:06.645012 (+ 789us) server_negotiation.cc:407] Connection header received
0809 18:54:06.645114 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:06.645115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:06.645125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:06.645153 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:06.645391 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.646084 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.647947 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.648491 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.648558 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.648567 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:06.649808 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0809 18:54:06.649855 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:54:06.650007 12501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18cbaa995cff426391aa42625199143a" candidate_uuid: "f4fee278fe954fe69f143dbf9e1f4060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" is_pre_election: true
I20260809 18:54:06.650151 12501 raft_consensus.cc:2454] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fee278fe954fe69f143dbf9e1f4060 in term 0.
I20260809 18:54:06.650170 12938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18cbaa995cff426391aa42625199143a" candidate_uuid: "f4fee278fe954fe69f143dbf9e1f4060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2628f27f356a4e848a8eb3cffba50c0f" is_pre_election: true
I20260809 18:54:06.650288 12938 raft_consensus.cc:2454] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4fee278fe954fe69f143dbf9e1f4060 in term 0.
I20260809 18:54:06.650406 12626 leader_election.cc:302] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4cc8bbd241540b4b93eb345cf1d4aed, f4fee278fe954fe69f143dbf9e1f4060; no voters:
I20260809 18:54:06.650584 13053 raft_consensus.cc:2781] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:54:06.650640 13053 raft_consensus.cc:482] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:54:06.650667 13053 raft_consensus.cc:3037] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:06.651505 13053 raft_consensus.cc:504] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } }
I20260809 18:54:06.651610 13053 leader_election.cc:288] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [CANDIDATE]: Term 1 election: Requested vote from peers 2628f27f356a4e848a8eb3cffba50c0f (127.23.218.195:40661), c4cc8bbd241540b4b93eb345cf1d4aed (127.23.218.193:46631)
I20260809 18:54:06.651777 12938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18cbaa995cff426391aa42625199143a" candidate_uuid: "f4fee278fe954fe69f143dbf9e1f4060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2628f27f356a4e848a8eb3cffba50c0f"
I20260809 18:54:06.651793 12501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18cbaa995cff426391aa42625199143a" candidate_uuid: "f4fee278fe954fe69f143dbf9e1f4060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed"
I20260809 18:54:06.651847 12938 raft_consensus.cc:3037] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:06.651847 12501 raft_consensus.cc:3037] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:06.652624 12501 raft_consensus.cc:2454] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fee278fe954fe69f143dbf9e1f4060 in term 1.
I20260809 18:54:06.652624 12938 raft_consensus.cc:2454] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4fee278fe954fe69f143dbf9e1f4060 in term 1.
I20260809 18:54:06.652786 12626 leader_election.cc:302] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4cc8bbd241540b4b93eb345cf1d4aed, f4fee278fe954fe69f143dbf9e1f4060; no voters:
I20260809 18:54:06.652854 13053 raft_consensus.cc:2781] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:06.652981 13053 raft_consensus.cc:686] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 1 LEADER]: Becoming Leader. State: Replica: f4fee278fe954fe69f143dbf9e1f4060, State: Running, Role: LEADER
I20260809 18:54:06.653069 13053 consensus_queue.cc:237] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } }
I20260809 18:54:06.653815 12267 catalog_manager.cc:5495] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4fee278fe954fe69f143dbf9e1f4060 (127.23.218.194). New cstate: current_term: 1 leader_uuid: "f4fee278fe954fe69f143dbf9e1f4060" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4fee278fe954fe69f143dbf9e1f4060" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 43167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 } health_report { overall_health: UNKNOWN } } }
Aug 09 18:54:06 dist-test-slave-6zbq krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301646, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for HTTP/127.23.218.212@KRBTEST.COM
I20260809 18:54:06.680969 13042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.677992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.23.218.194:43167)
0809 18:54:06.678067 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:06.678069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:06.678076 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:54:06.678107 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:06.678108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:06.678126 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:06.678189 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:54:06.678401 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.679006 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.679977 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:06.680235 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:06.680295 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:06.680494 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:54:06.680610 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:54:06.680687 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:06.680889 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 18:54:06.680908 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 18:54:06.683030 12501 raft_consensus.cc:1261] T 18cbaa995cff426391aa42625199143a P c4cc8bbd241540b4b93eb345cf1d4aed [term 1 FOLLOWER]: Refusing update from remote peer f4fee278fe954fe69f143dbf9e1f4060: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:06.683422 13053 consensus_queue.cc:1035] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4cc8bbd241540b4b93eb345cf1d4aed" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:06.685518 12938 raft_consensus.cc:1261] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [term 1 FOLLOWER]: Refusing update from remote peer f4fee278fe954fe69f143dbf9e1f4060: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:06.685837 13053 consensus_queue.cc:1035] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2628f27f356a4e848a8eb3cffba50c0f" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:06.689046 13068 mvcc.cc:205] Tried to move back new op lower bound from 7316691544811307008 to 7316691544691494912. Current Snapshot: MvccSnapshot[applied={T|T < 7316691544811307008}]
I20260809 18:54:06.690922 13072 mvcc.cc:205] Tried to move back new op lower bound from 7316691544811307008 to 7316691544691494912. Current Snapshot: MvccSnapshot[applied={T|T < 7316691544811307008}]
I20260809 18:54:06.691486 13070 mvcc.cc:205] Tried to move back new op lower bound from 7316691544811307008 to 7316691544691494912. Current Snapshot: MvccSnapshot[applied={T|T < 7316691544811307008}]
I20260809 18:54:06.696887 12266 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:06.696972 12266 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:06.697705 12266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d3e3cfef13b4df28956b268650f240e: Sending DeleteTablet for 3 replicas of tablet 18cbaa995cff426391aa42625199143a
I20260809 18:54:06.698087 12697 tablet_service.cc:1514] Processing DeleteTablet for tablet 18cbaa995cff426391aa42625199143a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:06 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:40368
I20260809 18:54:06.698087 12904 tablet_service.cc:1514] Processing DeleteTablet for tablet 18cbaa995cff426391aa42625199143a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:06 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:49472
I20260809 18:54:06.698295 12486 tablet_service.cc:1514] Processing DeleteTablet for tablet 18cbaa995cff426391aa42625199143a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:06 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:58324
I20260809 18:54:06.698724 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 12329
I20260809 18:54:06.705700 13086 tablet_replica.cc:330] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: stopping tablet replica
I20260809 18:54:06.705830 13086 raft_consensus.cc:2229] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:54:06.705971 13086 raft_consensus.cc:2258] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:06.709925 13086 ts_tablet_manager.cc:1905] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:06.711670 13086 ts_tablet_manager.cc:1918] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:54:06.718456 13088 tablet_replica.cc:330] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: stopping tablet replica
I20260809 18:54:06.718652 13088 raft_consensus.cc:2229] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:06.718838 13088 raft_consensus.cc:2258] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:06.719033 13086 log.cc:1196] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/18cbaa995cff426391aa42625199143a
I20260809 18:54:06.719350 13086 ts_tablet_manager.cc:1939] T 18cbaa995cff426391aa42625199143a P f4fee278fe954fe69f143dbf9e1f4060: Deleting consensus metadata
I20260809 18:54:06.719832 12245 catalog_manager.cc:4841] TS f4fee278fe954fe69f143dbf9e1f4060 (127.23.218.194:43167): tablet 18cbaa995cff426391aa42625199143a (table test-table [id=4a2db8f3be034d7ca2be1c4cb4efeb27]) successfully deleted
I20260809 18:54:06.720049 13088 ts_tablet_manager.cc:1905] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:06.721797 13088 ts_tablet_manager.cc:1918] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:54:06.722404 13088 log.cc:1196] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/18cbaa995cff426391aa42625199143a
I20260809 18:54:06.722726 13088 ts_tablet_manager.cc:1939] T 18cbaa995cff426391aa42625199143a P 2628f27f356a4e848a8eb3cffba50c0f: Deleting consensus metadata
I20260809 18:54:06.723176 12249 catalog_manager.cc:4841] TS 2628f27f356a4e848a8eb3cffba50c0f (127.23.218.195:40661): tablet 18cbaa995cff426391aa42625199143a (table test-table [id=4a2db8f3be034d7ca2be1c4cb4efeb27]) successfully deleted
W20260809 18:54:06.730731 12249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:46631 (error 108)
I20260809 18:54:06.731596 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 12600
I20260809 18:54:06.731662 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.731462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46631 (local address 127.0.0.1:58326)
0809 18:54:06.731570 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:06.731622 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:46631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 18:54:06.740526 12249 catalog_manager.cc:4584] TS c4cc8bbd241540b4b93eb345cf1d4aed (127.23.218.193:46631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 18cbaa995cff426391aa42625199143a: Network error: Client connection negotiation failed: client connection to 127.23.218.193:46631: connect: Connection refused (error 111)
I20260809 18:54:06.793113 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.792971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46631 (local address 127.0.0.1:58336)
0809 18:54:06.793026 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:06.793056 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:46631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 18:54:06.793529 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:06.793451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46631 (local address 127.0.0.1:58338)
0809 18:54:06.793483 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:06.793504 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:46631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 18:54:06.813071 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 12802
I20260809 18:54:06.829543 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 12210
I20260809 18:54:06.845701 24427 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 18:54:07.189440 24427 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 18:54:07.189527 24427 mini_ranger.cc:66] Stopping Ranger...
I20260809 18:54:07.532825 24427 mini_ranger.cc:68] Stopped Ranger
2026-08-09 18:54:07.532 UTC [10729] LOG: received smart shutdown request
2026-08-09 18:54:07.537 UTC [10729] LOG: background worker "logical replication launcher" (PID 10747) exited with exit code 1
2026-08-09 18:54:07.537 UTC [10735] LOG: shutting down
2026-08-09 18:54:07.657 UTC [10729] LOG: database system is shut down
2026-08-09T18:54:07Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44995 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:54:07 dist-test-slave-6zbq krb5kdc[13106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:54:07 dist-test-slave-6zbq krb5kdc[13106](info): set up 2 sockets
Aug 09 18:54:07 dist-test-slave-6zbq krb5kdc[13106](info): commencing operation
krb5kdc: starting...
W20260809 18:54:08.901711 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:54:08 dist-test-slave-6zbq krb5kdc[13106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:54:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:54:09.007028 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:46505
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:46505,127.23.218.253:46011,127.23.218.252:38891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:09.085202 13126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:09.085383 13126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:09.085412 13126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:09.086517 13126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:09.086555 13126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:09.086573 13126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:09.086589 13126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:09.086604 13126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:09.088025 13126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:46505,127.23.218.253:46011,127.23.218.252:38891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:46505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:09.088260 13126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:09.088440 13126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:09.092854 13133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:09.092897 13136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:09.092963 13138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:09.092870 13126 server_base.cc:1034] running on GCE node
I20260809 18:54:09.093241 13126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:09.093492 13126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:09.094623 13126 hybrid_clock.cc:648] HybridClock initialized: now 1786301649094604 us; error 30 us; skew 500 ppm
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:09.096774 13126 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:54:09.097180 13126 webserver.cc:460] Webserver started at http://127.23.218.254:44553/ using document root <none> and password file <none>
I20260809 18:54:09.097370 13126 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:09.097416 13126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:09.097532 13126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:09.098524 13126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "a1cef082acd646b8837a30eaa64ef10c"
format_stamp: "Formatted at 2026-08-09 18:54:09 on dist-test-slave-6zbq"
server_key: "5ba5cf22d0de441521961705135b1642"
server_key_iv: "da971ff84180256877bface5897b89c9"
server_key_version: "encryptionkey@0"
I20260809 18:54:09.098831 13126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "a1cef082acd646b8837a30eaa64ef10c"
format_stamp: "Formatted at 2026-08-09 18:54:09 on dist-test-slave-6zbq"
server_key: "5ba5cf22d0de441521961705135b1642"
server_key_iv: "da971ff84180256877bface5897b89c9"
server_key_version: "encryptionkey@0"
I20260809 18:54:09.099963 13126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:09.100580 13153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:09.100724 13126 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:09.100805 13126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "a1cef082acd646b8837a30eaa64ef10c"
format_stamp: "Formatted at 2026-08-09 18:54:09 on dist-test-slave-6zbq"
server_key: "5ba5cf22d0de441521961705135b1642"
server_key_iv: "da971ff84180256877bface5897b89c9"
server_key_version: "encryptionkey@0"
I20260809 18:54:09.100862 13126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:09.136837 13126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:09.138413 13126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:09.138538 13126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:09.142268 13126 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:46505
I20260809 18:54:09.142298 13243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:46505 every 8 connection(s)
I20260809 18:54:09.142620 13126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:54:09.143393 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:09.144747 13244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:09.151145 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 13126
I20260809 18:54:09.151253 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:54:09.151443 24427 external_mini_cluster.cc:1428] Setting key 718fe508faf46e3f0bbc3d2f39713c68
I20260809 18:54:09.153223 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.145327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.23.218.254:46505)
0809 18:54:09.145465 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.145474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.145574 (+ 100us) server_negotiation.cc:407] Connection header received
0809 18:54:09.146061 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.146064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.146084 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.146172 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.146691 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.147469 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.148266 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.148565 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.148656 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.150364 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.150370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.150371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.150377 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.151786 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.152143 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.152146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.152147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.152246 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.152420 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.152420 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.152421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.152664 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.152734 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.152877 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.153050 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260809 18:54:09.153230 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.145288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:46505 (local address 127.0.0.1:36212)
0809 18:54:09.145457 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.145494 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:09.145925 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:09.146182 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:09.146185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:09.146457 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:09.146607 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.146612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.147551 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.147552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.148194 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.148196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.148638 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.148639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.148716 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.149076 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:09.149084 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:09.150274 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:09.151876 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.151883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.151885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.152066 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.152302 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.152303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.152303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.152371 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.152724 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:09.152728 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:09.152776 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 18:54:09.152837 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 18:54:09.152967 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260809 18:54:09.153851 13244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:54:09.154153 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.154025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:46011 (local address 127.0.0.1:37982)
0809 18:54:09.154068 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.154132 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 18:54:09.154264 13157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
I20260809 18:54:09.154651 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.154556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:46011 (local address 127.0.0.1:37992)
0809 18:54:09.154606 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.154631 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 18:54:09.154747 13244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.253:46011: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.23.218.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab.
Entry for principal kudu/127.23.218.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab.
I20260809 18:54:09.172765 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.253:46011
--webserver_interface=127.23.218.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab
--principal=kudu/127.23.218.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:46505,127.23.218.253:46011,127.23.218.252:38891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:54:09.203933 13244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } attempt: 1
I20260809 18:54:09.204336 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.204230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:46011 (local address 127.0.0.1:37998)
0809 18:54:09.204280 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.204309 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 18:54:09.204659 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.204570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:46011 (local address 127.0.0.1:38000)
0809 18:54:09.204610 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.204635 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 18:54:09.204771 13244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.253:46011: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
W20260809 18:54:09.250802 13253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:09.250982 13253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:09.251011 13253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:09.252100 13253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:09.252138 13253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:09.252156 13253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:09.252171 13253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:09.252187 13253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:09.253600 13253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:46505,127.23.218.253:46011,127.23.218.252:38891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab
--principal=kudu/127.23.218.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.253:46011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.218.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:09.253824 13253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:09.254011 13253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:09.258380 13269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:09.258486 13253 server_base.cc:1034] running on GCE node
W20260809 18:54:09.258406 13266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:09.258406 13267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:09.258821 13253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:09.259079 13253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:09.260205 13253 hybrid_clock.cc:648] HybridClock initialized: now 1786301649260198 us; error 29 us; skew 500 ppm
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:09.262435 13253 init.cc:377] Logged in from keytab as kudu/127.23.218.253@KRBTEST.COM (short username kudu)
I20260809 18:54:09.262792 13253 webserver.cc:460] Webserver started at http://127.23.218.253:34045/ using document root <none> and password file <none>
I20260809 18:54:09.262964 13253 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:09.263000 13253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:09.263083 13253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:09.263962 13253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/data/instance:
uuid: "ee8c18caf9ec453185df63bcc0ac8fa2"
format_stamp: "Formatted at 2026-08-09 18:54:09 on dist-test-slave-6zbq"
server_key: "77aad14d8161c2f875645bf69e57a8f2"
server_key_iv: "2ed98eaad13188f574febe307bf147fb"
server_key_version: "encryptionkey@0"
I20260809 18:54:09.264245 13253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/wal/instance:
uuid: "ee8c18caf9ec453185df63bcc0ac8fa2"
format_stamp: "Formatted at 2026-08-09 18:54:09 on dist-test-slave-6zbq"
server_key: "77aad14d8161c2f875645bf69e57a8f2"
server_key_iv: "2ed98eaad13188f574febe307bf147fb"
server_key_version: "encryptionkey@0"
I20260809 18:54:09.265327 13253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:54:09.265940 13277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:09.266101 13253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:09.266180 13253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/wal
uuid: "ee8c18caf9ec453185df63bcc0ac8fa2"
format_stamp: "Formatted at 2026-08-09 18:54:09 on dist-test-slave-6zbq"
server_key: "77aad14d8161c2f875645bf69e57a8f2"
server_key_iv: "2ed98eaad13188f574febe307bf147fb"
server_key_version: "encryptionkey@0"
I20260809 18:54:09.266235 13253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:09.272890 13244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } attempt: 2
I20260809 18:54:09.273221 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.273089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:46011 (local address 127.0.0.1:38006)
0809 18:54:09.273143 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.273180 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 18:54:09.273504 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.273431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:46011 (local address 127.0.0.1:38020)
0809 18:54:09.273467 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.273488 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 18:54:09.273581 13244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.253:46011: Network error: Client connection negotiation failed: client connection to 127.23.218.253:46011: connect: Connection refused (error 111)
I20260809 18:54:09.287895 13253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:09.289435 13253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:09.289583 13253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:09.293206 13253 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.253:46011
I20260809 18:54:09.293256 13359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.253:46011 every 8 connection(s)
I20260809 18:54:09.293522 13253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
I20260809 18:54:09.294252 13363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:09.295578 13363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:54:09.296496 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 13253
I20260809 18:54:09.296607 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-1/wal/instance
I20260809 18:54:09.296759 24427 external_mini_cluster.cc:1428] Setting key 5d80fb67ab4be8d25f4e71dcb47d82d8
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:09.304425 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.296146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.23.218.254:46505)
0809 18:54:09.296190 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.296192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.296388 (+ 196us) server_negotiation.cc:407] Connection header received
0809 18:54:09.296856 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.296857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.296872 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.296923 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.297502 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.298184 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.299536 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.299819 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.299892 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.301708 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.301710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.301710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.301719 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.303156 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.303517 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.303518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.303519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.303592 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.303839 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.303840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.303840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.303973 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.304045 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.304253 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.304336 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 18:54:09.304487 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.296120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:46505 (local address 127.0.0.1:36222)
0809 18:54:09.296268 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.296305 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:09.296759 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:09.296998 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:09.297002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:09.297222 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:09.297399 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.297404 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.298292 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.298293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.299449 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.299452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.299898 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.299899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.300005 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.300405 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:09.300413 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:09.301617 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:09.303251 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.303257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.303259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.303452 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.303655 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.303656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.303656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.303781 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.304033 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:09.304036 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:09.304132 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 18:54:09.304219 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 18:54:09.304343 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260809 18:54:09.305001 13363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.218.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.252@KRBTEST.COM" created.
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
I20260809 18:54:09.311676 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.305197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:46011 (local address 127.0.0.1:38034)
0809 18:54:09.305242 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.305252 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:09.305297 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:09.305508 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:09.305509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:09.305647 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:09.305697 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.305699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.306546 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.306547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.307025 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.307026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.307450 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.307451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.307529 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.307874 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:09.307879 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:09.309109 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:09.310804 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.310805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.310805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.310949 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.311141 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.311141 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.311142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.311182 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.311456 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:09.311456 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:09.311486 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 18:54:09.311527 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 18:54:09.311600 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 18:54:09.311815 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.305248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38034 (local address 127.23.218.253:46011)
0809 18:54:09.305377 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.305381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.305398 (+ 17us) server_negotiation.cc:407] Connection header received
0809 18:54:09.305418 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.305420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.305432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.305488 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.305754 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.306483 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.307084 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.307397 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.307446 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.309208 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.309217 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.309217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.309224 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.310733 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.311020 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.311023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.311023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.311085 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.311218 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.311219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.311219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.311396 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.311479 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.311563 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.311669 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260809 18:54:09.312631 13363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:54:09.312888 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.312799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:41966)
0809 18:54:09.312837 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.312872 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 18:54:09.312996 13280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
I20260809 18:54:09.313326 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.313251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:41968)
0809 18:54:09.313284 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.313306 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 18:54:09.313408 13363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38891: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab.
Entry for principal kudu/127.23.218.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab.
I20260809 18:54:09.320474 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.252:38891
--webserver_interface=127.23.218.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab
--principal=kudu/127.23.218.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:46295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:46505,127.23.218.253:46011,127.23.218.252:38891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:54:09.362602 13363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } attempt: 1
I20260809 18:54:09.363044 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.362912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:41972)
0809 18:54:09.362970 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.363009 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 18:54:09.363365 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.363277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:41988)
0809 18:54:09.363319 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.363346 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 18:54:09.363459 13363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38891: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
I20260809 18:54:09.364718 13244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } attempt: 3
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
I20260809 18:54:09.371428 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.364981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:46011 (local address 127.0.0.1:38036)
0809 18:54:09.365032 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.365043 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:09.365091 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:09.365216 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:09.365217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:09.365349 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:09.365416 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.365418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.366324 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.366324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.366835 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.366837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.367268 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.367268 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.367321 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.367736 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:09.367740 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:09.368899 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:09.370526 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.370527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.370527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.370681 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.370877 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.370878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.370878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.370923 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.371183 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:09.371184 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:09.371241 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 18:54:09.371295 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 18:54:09.371366 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 18:54:09.371485 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.365019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38036 (local address 127.23.218.253:46011)
0809 18:54:09.365062 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.365064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.365080 (+ 16us) server_negotiation.cc:407] Connection header received
0809 18:54:09.365137 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.365138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.365150 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.365198 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.365471 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.366253 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.366900 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.367207 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.367272 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.368972 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.368974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.368974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.368981 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.370454 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.370751 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.370752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.370753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.370816 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.370967 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.370968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.370968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.371093 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.371154 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.371329 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.371404 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:54:09.371824 13244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:54:09.372125 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.372040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:42000)
0809 18:54:09.372079 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.372104 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 18:54:09.372402 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.372325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:42014)
0809 18:54:09.372361 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.372382 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 18:54:09.372489 13244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38891: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
W20260809 18:54:09.401476 13371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:09.401638 13371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:09.401659 13371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:09.402725 13371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:09.402758 13371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:09.402770 13371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:09.402781 13371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:09.402798 13371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:09.403573 13244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } attempt: 1
I20260809 18:54:09.403875 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.403782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:42022)
0809 18:54:09.403827 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.403856 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 18:54:09.404173 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.404098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:42032)
0809 18:54:09.404133 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.404156 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 18:54:09.404256 13244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38891: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
I20260809 18:54:09.404496 13371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:46295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:46505,127.23.218.253:46011,127.23.218.252:38891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab
--principal=kudu/127.23.218.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.252:38891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.218.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:09.404721 13371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:09.404912 13371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:09.409144 13383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:09.409169 13380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:09.409171 13379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:09.409327 13371 server_base.cc:1034] running on GCE node
I20260809 18:54:09.409547 13371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:09.409794 13371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:09.410923 13371 hybrid_clock.cc:648] HybridClock initialized: now 1786301649410911 us; error 25 us; skew 500 ppm
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:09.413080 13371 init.cc:377] Logged in from keytab as kudu/127.23.218.252@KRBTEST.COM (short username kudu)
I20260809 18:54:09.413452 13371 webserver.cc:460] Webserver started at http://127.23.218.252:33699/ using document root <none> and password file <none>
I20260809 18:54:09.413650 13371 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:09.413695 13371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:09.413939 13371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:09.415001 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/data/instance:
uuid: "4c84bf6ac6534cb0a44cdc858f6a3691"
format_stamp: "Formatted at 2026-08-09 18:54:09 on dist-test-slave-6zbq"
server_key: "96b09afda197ad480f7e67aab59d8e42"
server_key_iv: "6d0a31c0005c38e908d219c8767bfdda"
server_key_version: "encryptionkey@0"
I20260809 18:54:09.415303 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/wal/instance:
uuid: "4c84bf6ac6534cb0a44cdc858f6a3691"
format_stamp: "Formatted at 2026-08-09 18:54:09 on dist-test-slave-6zbq"
server_key: "96b09afda197ad480f7e67aab59d8e42"
server_key_iv: "6d0a31c0005c38e908d219c8767bfdda"
server_key_version: "encryptionkey@0"
I20260809 18:54:09.416445 13371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:54:09.417052 13395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:09.417191 13371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:09.417279 13371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/wal
uuid: "4c84bf6ac6534cb0a44cdc858f6a3691"
format_stamp: "Formatted at 2026-08-09 18:54:09 on dist-test-slave-6zbq"
server_key: "96b09afda197ad480f7e67aab59d8e42"
server_key_iv: "6d0a31c0005c38e908d219c8767bfdda"
server_key_version: "encryptionkey@0"
I20260809 18:54:09.417339 13371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:09.431597 13363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } attempt: 2
I20260809 18:54:09.431929 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.431814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:42040)
0809 18:54:09.431865 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.431899 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 18:54:09.432214 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.432140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:42054)
0809 18:54:09.432176 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.432197 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 18:54:09.432310 13363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38891: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38891: connect: Connection refused (error 111)
I20260809 18:54:09.445010 13371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:09.446882 13371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:09.447001 13371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:09.450464 13371 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.252:38891
I20260809 18:54:09.450505 13475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.252:38891 every 8 connection(s)
I20260809 18:54:09.450776 13371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
I20260809 18:54:09.451505 13480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:09.452785 13480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:54:09.454605 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 13371
I20260809 18:54:09.454715 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786301499543878-24427-0/minicluster-data/master-2/wal/instance
I20260809 18:54:09.454887 24427 external_mini_cluster.cc:1428] Setting key bc9ab0d78bbd876225544d809fb7a468
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
I20260809 18:54:09.461278 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.453340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.23.218.254:46505)
0809 18:54:09.453385 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.453387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.453551 (+ 164us) server_negotiation.cc:407] Connection header received
0809 18:54:09.453941 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.453942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.453950 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.453998 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.454559 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.455200 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.456187 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.456398 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.456455 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.458615 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.458616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.458617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.458624 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.459755 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.460101 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.460102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.460103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.460159 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.460404 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.460405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.460405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.460541 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.460613 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.461095 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.461185 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 18:54:09.461396 13483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.453311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:46505 (local address 127.0.0.1:36234)
0809 18:54:09.453451 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.453487 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:09.453876 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:09.454045 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:09.454048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:09.454264 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:09.454478 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.454483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.455282 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.455283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.456096 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.456099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.456470 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.456471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.456597 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.456988 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:09.456997 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:09.458539 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:09.459836 (+ 1297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.459843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.459844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.460037 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.460232 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.460232 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.460233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.460338 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.460799 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:09.460804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:09.460946 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 18:54:09.461051 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 18:54:09.461207 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260809 18:54:09.461866 13480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:54:09.463163 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.455352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42064 (local address 127.23.218.252:38891)
0809 18:54:09.455507 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.455511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.455527 (+ 16us) server_negotiation.cc:407] Connection header received
0809 18:54:09.455544 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.455545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.455563 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.455622 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.456164 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.456924 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.457861 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.458314 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.458421 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.460605 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.460608 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.460609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.460613 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.462149 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.462505 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.462508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.462508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.462564 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.462737 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.462737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.462738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.462845 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.462946 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.463031 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.463096 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:09.467729 13483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.462068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:46011 (local address 127.0.0.1:38044)
0809 18:54:09.462114 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.462125 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:09.462167 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:09.462288 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:09.462288 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:09.462430 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:09.462456 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.462457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.463211 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.463212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.463761 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.463764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.464171 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.464172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.464246 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.464536 (+ 290us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:09.464540 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:09.465472 (+ 932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:09.466859 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.466859 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.466860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.467031 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.467210 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.467210 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.467210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.467278 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.467466 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:09.467467 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:09.467507 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 18:54:09.467570 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 18:54:09.467646 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 18:54:09.467761 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.462117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.23.218.253:46011)
0809 18:54:09.462154 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.462155 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.462163 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:54:09.462217 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.462217 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.462227 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.462279 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.462524 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.463126 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.463834 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.464103 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.464149 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.465542 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.465544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.465544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.465551 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.466789 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.467105 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.467106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.467106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.467153 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.467331 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.467332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.467332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.467417 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.467458 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.467604 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.467672 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:54:09.468065 13480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:54:09.470322 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.463882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.23.218.254:46505)
0809 18:54:09.463925 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.463927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.463933 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:54:09.463950 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.463950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.463959 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.464007 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.464500 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.464937 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.465628 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.465805 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.465851 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.467873 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.467874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.467875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.467882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.469406 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.469708 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.469708 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.469709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.469759 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.469946 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.469946 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.469947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.470045 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.470164 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.470213 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.470269 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
I20260809 18:54:09.474598 13483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.468270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:42074)
0809 18:54:09.468300 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.468309 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:09.468344 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:09.468472 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:09.468473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:09.468604 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:09.468634 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.468636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.469726 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.469727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.470382 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.470385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.470925 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.470926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.470981 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.471288 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:09.471292 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:09.472315 (+ 1023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:09.473694 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.473695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.473695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.473840 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.473988 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.473989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.473989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.474041 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.474269 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:09.474270 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:09.474313 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 18:54:09.474409 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 18:54:09.474493 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 18:54:09.474661 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.468299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42074 (local address 127.23.218.252:38891)
0809 18:54:09.468327 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.468329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.468337 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:54:09.468407 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.468408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.468417 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.468463 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.468703 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.469395 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.470459 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.470730 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.470786 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.472382 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.472383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.472384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.472390 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.473624 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.473896 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.473897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.473897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.473939 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.474110 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.474111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.474111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.474207 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.474262 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.474426 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.474547 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":14}
I20260809 18:54:09.475873 13480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691: Bootstrap starting.
I20260809 18:54:09.476624 13480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:09.476830 13480 log.cc:824] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:09.477085 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.470951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.23.218.253:46011)
0809 18:54:09.470993 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.470994 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.471001 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:54:09.471018 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.471019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.471027 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.471065 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.471545 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.471957 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.472703 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.472891 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.472941 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.474931 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.474932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.474932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.474938 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.476130 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.476401 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.476402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.476403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.476451 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.476604 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.476604 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.476604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.476700 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.476850 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.476883 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.476942 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 18:54:09.477502 13480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691: No bootstrap required, opened a new log
I20260809 18:54:09.478638 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:54:09.479075 13480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } }
I20260809 18:54:09.479184 13480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:09.479203 13480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c84bf6ac6534cb0a44cdc858f6a3691, State: Initialized, Role: FOLLOWER
I20260809 18:54:09.479314 13480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } }
I20260809 18:54:09.479367 13244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } attempt: 2
I20260809 18:54:09.479584 13480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:09.480974 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
I20260809 18:54:09.481143 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:54:09.484118 13371 master_runner.cc:186] Error getting master registration for 127.23.218.254:46505: OK, instance_id { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" instance_seqno: 1786301649141245 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 18:54:09.486699 13371 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 18:54:09.487291 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.479845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:42086)
0809 18:54:09.479902 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.479910 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:09.479946 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:09.480181 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:09.480182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:09.480325 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:09.480365 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.480367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.481224 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.481225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.481685 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.481686 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.482057 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.482058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.482110 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.482480 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:09.482486 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:09.483644 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:09.485297 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.485298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.485299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.485503 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.485696 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.485697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.485697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.485751 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.486502 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:09.486503 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:09.486548 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 18:54:09.486900 (+ 352us) client_negotiation.cc:240] Negotiation successful
0809 18:54:09.486990 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 18:54:09.483688 13521 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:54:09.487505 13521 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:54:09.488054 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.480032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.23.218.252:38891)
0809 18:54:09.480065 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.480066 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.480073 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:54:09.480091 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.480092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.480100 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.480146 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.480445 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.481058 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.481763 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.482004 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.482057 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.483812 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.483813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.483813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.483819 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.485211 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.485577 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.485578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.485578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.485633 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.485817 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.485818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.485818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.486404 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.486470 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.487905 (+ 1435us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.487974 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:54:09.488477 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.479876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.23.218.253:46011)
0809 18:54:09.479906 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.479907 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.480344 (+ 437us) server_negotiation.cc:407] Connection header received
0809 18:54:09.480406 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.480407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.480415 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.480456 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.481152 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.481831 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.482805 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.483064 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.483126 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.485901 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.485903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.485903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.485909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.487353 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.487707 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.487708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.487709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.487776 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.487962 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.487962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.487963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.488069 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.488126 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.488320 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.488396 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 18:54:09.489431 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.479868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.23.218.254:46505)
0809 18:54:09.480002 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.480003 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.482027 (+ 2024us) server_negotiation.cc:407] Connection header received
0809 18:54:09.482331 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.482332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.482376 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.482404 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.484164 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.484713 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.485334 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.485594 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.485629 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.486891 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.486893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.486893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.486899 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.488274 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.488634 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.488635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.488635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.488685 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.488893 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.488894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.488894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.489014 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.489087 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.489281 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.489353 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260809 18:54:09.490641 13244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c: Bootstrap starting.
I20260809 18:54:09.491171 13244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:09.491427 13244 log.cc:824] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:09.492033 13244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c: No bootstrap required, opened a new log
I20260809 18:54:09.492133 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.480122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42102 (local address 127.23.218.252:38891)
0809 18:54:09.480544 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.480546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.480552 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:54:09.480569 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.480569 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.480577 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.480632 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.481241 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.481932 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.482842 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.483123 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.483187 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.489118 (+ 5931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.489119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.489119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.489126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.490708 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.491086 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.491087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.491087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.491150 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.491347 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.491348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.491348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.491460 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.491915 (+ 455us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.491955 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.492027 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":85,"threads_started":1}
I20260809 18:54:09.493400 13244 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } }
I20260809 18:54:09.493498 13244 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:09.493530 13244 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cef082acd646b8837a30eaa64ef10c, State: Initialized, Role: FOLLOWER
I20260809 18:54:09.493639 13244 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } }
I20260809 18:54:09.493989 13244 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:09.493978 13527 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
I20260809 18:54:09.494127 13527 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 18:54:09.495931 13540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:54:09.496025 13540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:54:09.496392 13126 master_runner.cc:186] Error getting master registration for 127.23.218.253:46011: OK, instance_id { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" instance_seqno: 1786301649292227 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 18:54:09.498615 13126 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 18:54:09.523459 13363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } attempt: 3
Aug 09 18:54:09 dist-test-slave-6zbq krb5kdc[13106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301649, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
I20260809 18:54:09.531136 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.523679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38891 (local address 127.0.0.1:42104)
0809 18:54:09.523731 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:09.523740 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:09.523773 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:09.523891 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:09.523891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:09.524046 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:09.524088 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.524090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.524972 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.524973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.525751 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:09.525754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.526221 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.526222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:09.526280 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.527019 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:09.527024 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:09.528403 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:09.530099 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.530100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.530101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.530320 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.530546 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:09.530547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:09.530547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:09.530609 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:09.530849 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:09.530849 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:09.530891 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 18:54:09.530955 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 18:54:09.531041 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 18:54:09.531153 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:09.523733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42104 (local address 127.23.218.252:38891)
0809 18:54:09.523786 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:09.523788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:09.523795 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:54:09.523816 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:09.523817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:09.523826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:09.523881 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:09.524152 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.524895 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.525833 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:09.526149 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:09.526216 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:09.528486 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:09.528488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:09.528488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:09.528495 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:09.530019 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.530415 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.530417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.530417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.530483 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:09.530663 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:09.530664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:09.530664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:09.530791 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:09.530853 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:09.530993 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 18:54:09.531067 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 18:54:09.532968 13363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2: Bootstrap starting.
I20260809 18:54:09.533643 13363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:09.533962 13363 log.cc:824] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:09.534798 13363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2: No bootstrap required, opened a new log
I20260809 18:54:09.536703 13363 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } }
I20260809 18:54:09.536849 13363 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:09.536916 13363 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8c18caf9ec453185df63bcc0ac8fa2, State: Initialized, Role: FOLLOWER
I20260809 18:54:09.537050 13363 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } }
I20260809 18:54:09.537335 13363 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:09.537300 13543 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
I20260809 18:54:09.537487 13543 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 18:54:09.539448 13562 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:54:09.539546 13562 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 18:54:09.540099 13285 tablet.cc:2369] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:54:09.540989 13253 master_runner.cc:263] No leader master found from master ee8c18caf9ec453185df63bcc0ac8fa2
I20260809 18:54:09.541034 13253 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 18:54:09.592434 13527 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:09.592540 13527 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } }
I20260809 18:54:09.592742 13527 leader_election.cc:288] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee8c18caf9ec453185df63bcc0ac8fa2 (127.23.218.253:46011), 4c84bf6ac6534cb0a44cdc858f6a3691 (127.23.218.252:38891)
I20260809 18:54:09.593158 13437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1cef082acd646b8837a30eaa64ef10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" is_pre_election: true
I20260809 18:54:09.593307 13437 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cef082acd646b8837a30eaa64ef10c in term 0.
I20260809 18:54:09.593619 13319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1cef082acd646b8837a30eaa64ef10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" is_pre_election: true
I20260809 18:54:09.593822 13319 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1cef082acd646b8837a30eaa64ef10c in term 0.
I20260809 18:54:09.594214 13157 leader_election.cc:302] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c84bf6ac6534cb0a44cdc858f6a3691, a1cef082acd646b8837a30eaa64ef10c; no voters:
I20260809 18:54:09.594581 13527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:54:09.594653 13527 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:54:09.594676 13527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:09.595656 13527 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } }
I20260809 18:54:09.595769 13527 leader_election.cc:288] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [CANDIDATE]: Term 1 election: Requested vote from peers ee8c18caf9ec453185df63bcc0ac8fa2 (127.23.218.253:46011), 4c84bf6ac6534cb0a44cdc858f6a3691 (127.23.218.252:38891)
I20260809 18:54:09.596158 13437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1cef082acd646b8837a30eaa64ef10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691"
I20260809 18:54:09.596237 13437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:09.596983 13437 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cef082acd646b8837a30eaa64ef10c in term 1.
I20260809 18:54:09.597185 13319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a1cef082acd646b8837a30eaa64ef10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2"
I20260809 18:54:09.597257 13319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:09.598063 13319 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1cef082acd646b8837a30eaa64ef10c in term 1.
I20260809 18:54:09.598271 13157 leader_election.cc:302] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1cef082acd646b8837a30eaa64ef10c, ee8c18caf9ec453185df63bcc0ac8fa2; no voters:
I20260809 18:54:09.598443 13527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:09.598604 13527 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [term 1 LEADER]: Becoming Leader. State: Replica: a1cef082acd646b8837a30eaa64ef10c, State: Running, Role: LEADER
I20260809 18:54:09.598687 13527 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } }
I20260809 18:54:09.599018 13527 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1cef082acd646b8837a30eaa64ef10c. Latest consensus state: current_term: 1 leader_uuid: "a1cef082acd646b8837a30eaa64ef10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
I20260809 18:54:09.599083 13527 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:09.599386 13568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:54:09.599623 13568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:54:09.601923 13437 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [term 1 FOLLOWER]: Refusing update from remote peer a1cef082acd646b8837a30eaa64ef10c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:09.602160 13319 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [term 1 FOLLOWER]: Refusing update from remote peer a1cef082acd646b8837a30eaa64ef10c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:09.603205 13527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:09.604746 13543 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1cef082acd646b8837a30eaa64ef10c. Latest consensus state: current_term: 1 leader_uuid: "a1cef082acd646b8837a30eaa64ef10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
I20260809 18:54:09.604826 13543 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:54:09.605062 13527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:09.606233 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1cef082acd646b8837a30eaa64ef10c. Latest consensus state: current_term: 1 leader_uuid: "a1cef082acd646b8837a30eaa64ef10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
I20260809 18:54:09.606312 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:54:09.608664 13527 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1cef082acd646b8837a30eaa64ef10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
I20260809 18:54:09.608736 13527 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:09.609160 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1cef082acd646b8837a30eaa64ef10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
I20260809 18:54:09.609230 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:54:09.609459 13572 mvcc.cc:205] Tried to move back new op lower bound from 7316691556766208000 to 7316691556756578304. Current Snapshot: MvccSnapshot[applied={T|T < 7316691556766208000}]
I20260809 18:54:09.609580 13575 mvcc.cc:205] Tried to move back new op lower bound from 7316691556766208000 to 7316691556756578304. Current Snapshot: MvccSnapshot[applied={T|T < 7316691556766208000}]
I20260809 18:54:09.609975 13527 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1cef082acd646b8837a30eaa64ef10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
I20260809 18:54:09.610044 13527 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:09.610543 13543 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1cef082acd646b8837a30eaa64ef10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1cef082acd646b8837a30eaa64ef10c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46505 } } peers { permanent_uuid: "ee8c18caf9ec453185df63bcc0ac8fa2" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 46011 } } peers { permanent_uuid: "4c84bf6ac6534cb0a44cdc858f6a3691" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38891 } } }
I20260809 18:54:09.610625 13543 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:54:09.610862 13573 mvcc.cc:205] Tried to move back new op lower bound from 7316691556766208000 to 7316691556756578304. Current Snapshot: MvccSnapshot[applied={T|T < 7316691556766208000}]
I20260809 18:54:09.611760 13568 catalog_manager.cc:1347] Generated new cluster ID: b9e0116de230405cb4eb9a61b056bbc8
I20260809 18:54:09.611801 13568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:54:09.623282 13568 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:54:09.623983 13568 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:54:09.632439 13568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1cef082acd646b8837a30eaa64ef10c: Generated new TSK 0
I20260809 18:54:09.632603 13568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:54:10.487931 13521 catalog_manager.cc:1259] Loaded cluster ID: b9e0116de230405cb4eb9a61b056bbc8
I20260809 18:54:10.487993 13521 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691: loading cluster ID for follower catalog manager: success
I20260809 18:54:10.488782 13521 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691: acquiring CA information for follower catalog manager: success
I20260809 18:54:10.489084 13521 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4c84bf6ac6534cb0a44cdc858f6a3691: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 18:54:10.539937 13562 catalog_manager.cc:1259] Loaded cluster ID: b9e0116de230405cb4eb9a61b056bbc8
I20260809 18:54:10.539989 13562 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2: loading cluster ID for follower catalog manager: success
I20260809 18:54:10.540747 13562 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2: acquiring CA information for follower catalog manager: success
I20260809 18:54:10.541045 13562 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee8c18caf9ec453185df63bcc0ac8fa2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 18:54:10.680881 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 13126
I20260809 18:54:10.691960 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 13253
I20260809 18:54:10.707737 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 13371
2026-08-09T18:54:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2959 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:54:10 dist-test-slave-6zbq krb5kdc[13596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:54:10 dist-test-slave-6zbq krb5kdc[13596](info): set up 2 sockets
Aug 09 18:54:10 dist-test-slave-6zbq krb5kdc[13596](info): commencing operation
krb5kdc: starting...
W20260809 18:54:11.826452 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:54:11 dist-test-slave-6zbq krb5kdc[13596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:54:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:54:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:54:11.931088 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:44391
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:35753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:44391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:12.008632 13617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:12.008806 13617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:12.008827 13617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:12.009928 13617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:12.009966 13617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:12.009979 13617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:12.009990 13617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:12.010000 13617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:12.011445 13617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:35753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:44391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:44391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:12.011655 13617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:12.011850 13617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:12.016288 13625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:12.016336 13617 server_base.cc:1034] running on GCE node
W20260809 18:54:12.016300 13624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:12.016300 13628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:12.016708 13617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:12.016960 13617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:12.018083 13617 hybrid_clock.cc:648] HybridClock initialized: now 1786301652018074 us; error 27 us; skew 500 ppm
Aug 09 18:54:12 dist-test-slave-6zbq krb5kdc[13596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301652, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:12.020274 13617 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:54:12.020663 13617 webserver.cc:460] Webserver started at http://127.23.218.254:37409/ using document root <none> and password file <none>
I20260809 18:54:12.020867 13617 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:12.020912 13617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:12.021027 13617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:12.021971 13617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "f5e9d6220e9e46188fa93977133949d7"
format_stamp: "Formatted at 2026-08-09 18:54:12 on dist-test-slave-6zbq"
server_key: "c2deb3822949a6aa0e228539251ebe94"
server_key_iv: "b863cfa8ad44a38a53ec70a4de71e9c0"
server_key_version: "encryptionkey@0"
I20260809 18:54:12.022286 13617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "f5e9d6220e9e46188fa93977133949d7"
format_stamp: "Formatted at 2026-08-09 18:54:12 on dist-test-slave-6zbq"
server_key: "c2deb3822949a6aa0e228539251ebe94"
server_key_iv: "b863cfa8ad44a38a53ec70a4de71e9c0"
server_key_version: "encryptionkey@0"
I20260809 18:54:12.023435 13617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:12.024046 13640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:12.024188 13617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:12.024277 13617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "f5e9d6220e9e46188fa93977133949d7"
format_stamp: "Formatted at 2026-08-09 18:54:12 on dist-test-slave-6zbq"
server_key: "c2deb3822949a6aa0e228539251ebe94"
server_key_iv: "b863cfa8ad44a38a53ec70a4de71e9c0"
server_key_version: "encryptionkey@0"
I20260809 18:54:12.024339 13617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:12.039484 13617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:12.040966 13617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:12.041100 13617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:12.044838 13617 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:44391
I20260809 18:54:12.044866 13731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:44391 every 8 connection(s)
I20260809 18:54:12.045151 13617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:54:12.045902 13732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:12.048177 13732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7: Bootstrap starting.
I20260809 18:54:12.048707 13732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:12.048931 13732 log.cc:824] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:12.049523 13732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7: No bootstrap required, opened a new log
I20260809 18:54:12.051093 13732 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e9d6220e9e46188fa93977133949d7" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44391 } }
I20260809 18:54:12.051222 13732 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:12.051244 13732 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5e9d6220e9e46188fa93977133949d7, State: Initialized, Role: FOLLOWER
I20260809 18:54:12.051338 13732 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e9d6220e9e46188fa93977133949d7" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44391 } }
I20260809 18:54:12.051385 13732 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:54:12.051409 13732 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:54:12.051438 13732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:12.051903 13732 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e9d6220e9e46188fa93977133949d7" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44391 } }
I20260809 18:54:12.051986 13732 leader_election.cc:302] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5e9d6220e9e46188fa93977133949d7; no voters:
I20260809 18:54:12.052104 13732 leader_election.cc:288] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:54:12.052145 13735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:12.052268 13735 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [term 1 LEADER]: Becoming Leader. State: Replica: f5e9d6220e9e46188fa93977133949d7, State: Running, Role: LEADER
I20260809 18:54:12.052335 13732 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:12.052385 13735 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e9d6220e9e46188fa93977133949d7" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44391 } }
I20260809 18:54:12.052734 13741 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5e9d6220e9e46188fa93977133949d7. Latest consensus state: current_term: 1 leader_uuid: "f5e9d6220e9e46188fa93977133949d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e9d6220e9e46188fa93977133949d7" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44391 } } }
I20260809 18:54:12.052839 13739 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5e9d6220e9e46188fa93977133949d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e9d6220e9e46188fa93977133949d7" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 44391 } } }
I20260809 18:54:12.052868 13741 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:12.052944 13739 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7 [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:12.053296 13755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:54:12.053757 13755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:54:12.054726 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 13617
I20260809 18:54:12.054827 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:54:12.055003 24427 external_mini_cluster.cc:1428] Setting key e8f499a803638c802408af130f3494be
I20260809 18:54:12.058683 13755 catalog_manager.cc:1347] Generated new cluster ID: 0f4767ca3ab94ad2948cace028c316b1
I20260809 18:54:12.058736 13755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:54:12 dist-test-slave-6zbq krb5kdc[13596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:12.069304 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:12.055368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53002 (local address 127.23.218.254:44391)
0809 18:54:12.058949 (+ 3581us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:12.058959 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:12.058992 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:54:12.059106 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:12.059114 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:12.059168 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:12.059303 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:12.060014 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:12.061741 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:12.062723 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:12.063323 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:12.063430 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:12.066020 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:12.066024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:12.066024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:12.066036 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:12.067810 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:12.068181 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:12.068183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:12.068184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:12.068277 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:12.068455 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:12.068456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:12.068456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:12.068725 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:12.068861 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:12.068965 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 18:54:12.069086 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3540,"thread_start_us":75,"threads_started":1}
I20260809 18:54:12.070230 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:54:12.075865 13755 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:54:12.076324 13755 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:54:12.080955 13755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5e9d6220e9e46188fa93977133949d7: Generated new TSK 0
I20260809 18:54:12.081103 13755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:54:12.123693 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 13617
2026-08-09T18:54:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1419 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:54:12 dist-test-slave-6zbq krb5kdc[13772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:54:12 dist-test-slave-6zbq krb5kdc[13772](info): set up 2 sockets
Aug 09 18:54:12 dist-test-slave-6zbq krb5kdc[13772](info): commencing operation
krb5kdc: starting...
W20260809 18:54:13.249594 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:54:13 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:54:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:54:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:54:13.354576 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38173
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:13.431532 13794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:13.431701 13794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:13.431732 13794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:13.432806 13794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:13.432842 13794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:13.432861 13794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:13.432876 13794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:13.432893 13794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:13.434317 13794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:13.434551 13794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:13.434746 13794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:13.439059 13807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:13.439142 13794 server_base.cc:1034] running on GCE node
W20260809 18:54:13.439081 13802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:13.439152 13805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:13.439486 13794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:13.439742 13794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:13.440871 13794 hybrid_clock.cc:648] HybridClock initialized: now 1786301653440862 us; error 27 us; skew 500 ppm
Aug 09 18:54:13 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301653, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:13.443157 13794 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:54:13.443517 13794 webserver.cc:460] Webserver started at http://127.23.218.254:35487/ using document root <none> and password file <none>
I20260809 18:54:13.443717 13794 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:13.443768 13794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:13.443884 13794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:13.444792 13794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "a0b3e4d6a65f49299690e11d502377e8"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "1e06275f14aac70f319854cbfd53b907"
server_key_iv: "d1d10ee465fbd3ebfb7c7e4acd6ce8ec"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.445083 13794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "a0b3e4d6a65f49299690e11d502377e8"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "1e06275f14aac70f319854cbfd53b907"
server_key_iv: "d1d10ee465fbd3ebfb7c7e4acd6ce8ec"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.446192 13794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:54:13.446870 13819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.447016 13794 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.447093 13794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "a0b3e4d6a65f49299690e11d502377e8"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "1e06275f14aac70f319854cbfd53b907"
server_key_iv: "d1d10ee465fbd3ebfb7c7e4acd6ce8ec"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.447156 13794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:13.453711 13794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:13.455611 13794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:13.455749 13794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:13.459286 13794 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:38173
I20260809 18:54:13.459313 13903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:38173 every 8 connection(s)
I20260809 18:54:13.459595 13794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:54:13.460474 13904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:13.462491 13904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Bootstrap starting.
I20260809 18:54:13.462991 13904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:13.463224 13904 log.cc:824] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:13.463797 13904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: No bootstrap required, opened a new log
I20260809 18:54:13.465287 13904 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:13.465415 13904 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:13.465442 13904 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b3e4d6a65f49299690e11d502377e8, State: Initialized, Role: FOLLOWER
I20260809 18:54:13.465541 13904 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:13.465615 13904 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:54:13.465643 13904 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:54:13.465675 13904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:13.466426 13904 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:13.466511 13904 leader_election.cc:302] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0b3e4d6a65f49299690e11d502377e8; no voters:
I20260809 18:54:13.466621 13904 leader_election.cc:288] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:54:13.466656 13909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:13.466828 13904 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:13.466848 13909 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 1 LEADER]: Becoming Leader. State: Replica: a0b3e4d6a65f49299690e11d502377e8, State: Running, Role: LEADER
I20260809 18:54:13.466990 13909 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:13.467368 13913 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0b3e4d6a65f49299690e11d502377e8. Latest consensus state: current_term: 1 leader_uuid: "a0b3e4d6a65f49299690e11d502377e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } } }
I20260809 18:54:13.467370 13910 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0b3e4d6a65f49299690e11d502377e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } } }
I20260809 18:54:13.467540 13910 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:13.467532 13913 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:13.467918 13925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:54:13.468209 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 13794
I20260809 18:54:13.468336 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:54:13.468412 13925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:54:13.468542 24427 external_mini_cluster.cc:1428] Setting key 342c0d753e80ed251bb27ee1d779932d
I20260809 18:54:13.469890 13925 catalog_manager.cc:1347] Generated new cluster ID: c73d8d5399a14430850eea3b74832051
I20260809 18:54:13.469931 13925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:54:13 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:13.478499 13925 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:54:13.478956 13925 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:54:13.480854 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.468967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57364 (local address 127.23.218.254:38173)
0809 18:54:13.469125 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:13.469135 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:13.470626 (+ 1491us) server_negotiation.cc:407] Connection header received
0809 18:54:13.470789 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:13.470796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:13.470848 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:13.470978 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:13.471693 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.473038 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.474079 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.474679 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.474781 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.477412 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:13.477416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:13.477416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:13.477428 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:13.479368 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.479717 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.479720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.479720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.479821 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.479989 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.479990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.479990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.480256 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:13.480384 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:13.480479 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 18:54:13.480627 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260809 18:54:13.482837 13925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Generated new TSK 0
I20260809 18:54:13.482987 13925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:54:13.506405 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38173
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:13.585101 13942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:13.585280 13942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:13.585311 13942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:13.586443 13942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:13.586481 13942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:13.586517 13942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:54:13.587939 13942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:13.588150 13942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:13.588344 13942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:13.592895 13951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:13.593014 13942 server_base.cc:1034] running on GCE node
W20260809 18:54:13.592880 13954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:13.592881 13950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:13.593307 13942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:13.593510 13942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:13.594636 13942 hybrid_clock.cc:648] HybridClock initialized: now 1786301653594630 us; error 33 us; skew 500 ppm
Aug 09 18:54:13 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301653, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:13.596895 13942 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:54:13.597282 13942 webserver.cc:460] Webserver started at http://127.23.218.193:44619/ using document root <none> and password file <none>
I20260809 18:54:13.597484 13942 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:13.597528 13942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:13.597649 13942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:13.598631 13942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "20b4a8593d9740aa9b3100d2db8b979c"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "186f4908c61ed6e95f7a584fd3f261e9"
server_key_iv: "85dbedbc03dd7cc12fd3249d5c3cbfaf"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.598944 13942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "20b4a8593d9740aa9b3100d2db8b979c"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "186f4908c61ed6e95f7a584fd3f261e9"
server_key_iv: "85dbedbc03dd7cc12fd3249d5c3cbfaf"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.600111 13942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:54:13.600757 13965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.600895 13942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:13.600971 13942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "20b4a8593d9740aa9b3100d2db8b979c"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "186f4908c61ed6e95f7a584fd3f261e9"
server_key_iv: "85dbedbc03dd7cc12fd3249d5c3cbfaf"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.601034 13942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:13.614734 13942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:13.616251 13942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:13.616366 13942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:13.616552 13942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:13.616808 13942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:13.616839 13942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.616858 13942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:13.616881 13942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.622119 13942 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:39791
I20260809 18:54:13.622179 14161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:39791 every 8 connection(s)
I20260809 18:54:13.622478 13942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
Aug 09 18:54:13 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301653, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:13.630069 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.623091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:49465 (local address 127.23.218.254:38173)
0809 18:54:13.623165 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:13.623167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:13.623321 (+ 154us) server_negotiation.cc:407] Connection header received
0809 18:54:13.623737 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:13.623738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:13.623748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:13.623790 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:13.624599 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.625068 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.625948 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.626148 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.626191 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.628043 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:13.628044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:13.628044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:13.628050 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:13.629088 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.629425 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.629425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.629426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.629467 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.629635 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.629636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.629636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.629721 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:13.629763 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:13.629945 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 18:54:13.630008 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 18:54:13.630231 14169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.623054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:49465)
0809 18:54:13.623202 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:13.623235 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:13.623661 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:13.623845 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:13.623848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:13.624354 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:13.624522 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.624527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.625140 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.625140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.625886 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.625888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.626200 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.626200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.626293 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.626728 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:13.626736 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:13.627963 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:13.629167 (+ 1204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:13.629173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:13.629174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:13.629370 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:13.629509 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:13.629510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:13.629510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:13.629595 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:13.629760 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:13.629762 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:13.629857 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 18:54:13.629928 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 18:54:13.630035 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260809 18:54:13.630723 14165 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38173
I20260809 18:54:13.630748 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 13942
I20260809 18:54:13.630807 14165 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:13.630836 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:54:13.630975 14165 heartbeater.cc:511] Master 127.23.218.254:38173 requested a full tablet report, sending...
I20260809 18:54:13.631014 24427 external_mini_cluster.cc:1428] Setting key 32456322ec34fcc375507265f9d84bc3
I20260809 18:54:13.631500 13851 ts_manager.cc:194] Registered new tserver with Master: 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193:39791)
I20260809 18:54:13.632251 13851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:49465
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:54:13.652559 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38173
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:13.730263 14174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:13.730456 14174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:13.730479 14174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:13.731523 14174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:13.731554 14174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:13.731578 14174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:54:13.732923 14174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:13.733088 14174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:13.733254 14174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:13.737741 14183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:13.737756 14182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:13.737790 14174 server_base.cc:1034] running on GCE node
W20260809 18:54:13.737896 14186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:13.738093 14174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:13.738307 14174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:13.739459 14174 hybrid_clock.cc:648] HybridClock initialized: now 1786301653739444 us; error 33 us; skew 500 ppm
Aug 09 18:54:13 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301653, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:13.746688 14174 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:54:13.747087 14174 webserver.cc:460] Webserver started at http://127.23.218.194:39091/ using document root <none> and password file <none>
I20260809 18:54:13.747277 14174 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:13.747323 14174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:13.747442 14174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:13.748404 14174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "723df2676ee847a9b8ca07b399413359"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "ed8eba22cbec483cd1dd0a1ba5a7a666"
server_key_iv: "fe06cb09640780345a0f7bdf8e8dfcc0"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.748711 14174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "723df2676ee847a9b8ca07b399413359"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "ed8eba22cbec483cd1dd0a1ba5a7a666"
server_key_iv: "fe06cb09640780345a0f7bdf8e8dfcc0"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.749969 14174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:54:13.750650 14197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.750815 14174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.750907 14174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "723df2676ee847a9b8ca07b399413359"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "ed8eba22cbec483cd1dd0a1ba5a7a666"
server_key_iv: "fe06cb09640780345a0f7bdf8e8dfcc0"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.750975 14174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:13.768150 14174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:13.770067 14174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:13.770186 14174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:13.770402 14174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:13.770676 14174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:13.770715 14174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.770735 14174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:13.770756 14174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.775933 14174 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:38725
I20260809 18:54:13.776000 14374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:38725 every 8 connection(s)
I20260809 18:54:13.776245 14174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:54:13.776592 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 14174
I20260809 18:54:13.776696 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:54:13.776892 24427 external_mini_cluster.cc:1428] Setting key c7a49008e1c66216fbf720318f8d8c4c
Aug 09 18:54:13 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301653, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
I20260809 18:54:13.784901 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.776963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:50269 (local address 127.23.218.254:38173)
0809 18:54:13.777005 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:13.777007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:13.777231 (+ 224us) server_negotiation.cc:407] Connection header received
0809 18:54:13.777801 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:13.777802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:13.777815 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:13.777864 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:13.778812 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.779303 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.780239 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.780444 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.780497 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.782457 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:13.782458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:13.782458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:13.782466 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:13.783680 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.784036 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.784037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.784037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.784093 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.784308 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.784308 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.784309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.784420 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:13.784476 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:13.784728 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 18:54:13.784812 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:54:13.785034 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.776948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:50269)
0809 18:54:13.777123 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:13.777167 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:13.777700 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:13.777929 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:13.777933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:13.778485 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:13.778728 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.778733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.779385 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.779386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.780159 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.780161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.780513 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.780514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.780644 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.781047 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:13.781055 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:13.782375 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:13.783752 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:13.783759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:13.783761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:13.783969 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:13.784149 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:13.784149 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:13.784150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:13.784251 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:13.784474 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:13.784481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:13.784610 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 18:54:13.784696 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 18:54:13.784841 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260809 18:54:13.785437 14375 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38173
I20260809 18:54:13.785521 14375 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:13.785674 14375 heartbeater.cc:511] Master 127.23.218.254:38173 requested a full tablet report, sending...
I20260809 18:54:13.786011 13851 ts_manager.cc:194] Registered new tserver with Master: 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725)
I20260809 18:54:13.786334 13851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:50269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:54:13.798372 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38173
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:13.877173 14389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:13.877354 14389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:13.877384 14389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:13.878494 14389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:13.878530 14389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:13.878566 14389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:54:13.879982 14389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:13.880208 14389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:13.880395 14389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:13.884833 14399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:13.884882 14389 server_base.cc:1034] running on GCE node
W20260809 18:54:13.884847 14400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:13.884862 14402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:13.885229 14389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:13.885455 14389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:13.886592 14389 hybrid_clock.cc:648] HybridClock initialized: now 1786301653886574 us; error 33 us; skew 500 ppm
Aug 09 18:54:13 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301653, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:13.888770 14389 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:54:13.889146 14389 webserver.cc:460] Webserver started at http://127.23.218.195:40211/ using document root <none> and password file <none>
I20260809 18:54:13.889343 14389 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:13.889389 14389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:13.889504 14389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:13.890458 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "57e0b19e67d448728b16a8146c33ada6"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "30538104bd5fdcf0c9337f56c64b9efd"
server_key_iv: "ee945a8a6611f0be433df1db59192aa2"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.890777 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "57e0b19e67d448728b16a8146c33ada6"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "30538104bd5fdcf0c9337f56c64b9efd"
server_key_iv: "ee945a8a6611f0be433df1db59192aa2"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.891938 14389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:13.892540 14412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.892683 14389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:13.892762 14389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "57e0b19e67d448728b16a8146c33ada6"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "30538104bd5fdcf0c9337f56c64b9efd"
server_key_iv: "ee945a8a6611f0be433df1db59192aa2"
server_key_version: "encryptionkey@0"
I20260809 18:54:13.892822 14389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:13.919597 14389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:13.921181 14389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:13.921303 14389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:13.921485 14389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:13.921757 14389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:13.921789 14389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.921815 14389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:13.921838 14389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:13.927101 14389 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:44939
I20260809 18:54:13.927137 14606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:44939 every 8 connection(s)
I20260809 18:54:13.927404 14389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:54:13.932349 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 14389
I20260809 18:54:13.932454 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:54:13.932628 24427 external_mini_cluster.cc:1428] Setting key 1a79ab2e9775f6dae319557cec61b4d7
Aug 09 18:54:13 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301653, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:13.935567 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.928026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:37713 (local address 127.23.218.254:38173)
0809 18:54:13.928079 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:13.928081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:13.928257 (+ 176us) server_negotiation.cc:407] Connection header received
0809 18:54:13.928726 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:13.928728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:13.928740 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:13.928789 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:13.929709 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.930199 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.931190 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.931387 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.931442 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.933460 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:13.933461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:13.933461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:13.933469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:13.934578 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.934915 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.934915 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.934916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.934961 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.935128 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.935128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.935128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.935220 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:13.935264 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:13.935452 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 18:54:13.935506 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 18:54:13.935726 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.928002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:37713)
0809 18:54:13.928148 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:13.928181 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:13.928626 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:13.928866 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:13.928869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:13.929435 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:13.929615 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.929620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.930282 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.930283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.931104 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.931106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.931455 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.931456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.931566 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.931989 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:13.931997 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:13.933374 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:13.934644 (+ 1270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:13.934649 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:13.934651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:13.934859 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:13.935003 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:13.935004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:13.935004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:13.935087 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:13.935261 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:13.935263 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:13.935361 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 18:54:13.935434 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 18:54:13.935543 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260809 18:54:13.936120 14607 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38173
I20260809 18:54:13.936194 14607 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:13.936331 14607 heartbeater.cc:511] Master 127.23.218.254:38173 requested a full tablet report, sending...
I20260809 18:54:13.936618 13851 ts_manager.cc:194] Registered new tserver with Master: 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939)
I20260809 18:54:13.936955 13851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:37713
I20260809 18:54:13.944181 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:54:13.950635 13937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.945428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57376 (local address 127.23.218.254:38173)
0809 18:54:13.945478 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:13.945480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:13.945585 (+ 105us) server_negotiation.cc:407] Connection header received
0809 18:54:13.945627 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:13.945628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:13.945636 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:13.945678 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:13.946172 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.946691 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.947358 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.947537 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.947585 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.948604 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:13.948604 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:13.948605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:13.948610 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:13.949714 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.950001 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.950001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.950002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.950048 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:13.950200 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:13.950201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:13.950201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:13.950366 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:13.950444 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:13.950520 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 18:54:13.950577 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 18:54:13.952005 13851 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:54:13.953577 13851 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:54:13.959622 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.955561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:38725 (local address 127.0.0.1:34746)
0809 18:54:13.955897 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:13.955937 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:13.956010 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:13.956271 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:13.956274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:13.956276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:13.956336 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.956338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.957750 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.957751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.958620 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.958623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.959309 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.959311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.959395 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.959408 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 18:54:13.959479 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 18:54:13.959532 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":3,"thread_start_us":55,"threads_started":1}
I20260809 18:54:13.959627 14625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.955727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.23.218.194:38725)
0809 18:54:13.955876 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:13.955881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:13.956001 (+ 120us) server_negotiation.cc:407] Connection header received
0809 18:54:13.956091 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:13.956094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:13.956110 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:13.956178 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:13.956413 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.957642 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.958694 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.959176 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.959261 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.959316 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:13.959500 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 18:54:13.959544 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260809 18:54:13.959721 14624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.955631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.23.218.195:44939)
0809 18:54:13.955822 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:13.955828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:13.956006 (+ 178us) server_negotiation.cc:407] Connection header received
0809 18:54:13.956237 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:13.956240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:13.956256 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:13.956324 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:13.956529 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.957214 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.958479 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.959296 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.959347 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.959401 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:13.959600 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 18:54:13.959641 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260809 18:54:13.959905 14622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.955561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:44939 (local address 127.0.0.1:43190)
0809 18:54:13.955898 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:13.955936 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:13.956157 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:13.956392 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:13.956393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:13.956395 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:13.956448 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.956450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.957306 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.957307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.958393 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.958396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.959461 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.959462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.959515 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.959518 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:54:13.959799 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 18:54:13.959846 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":2,"thread_start_us":55,"threads_started":1}
I20260809 18:54:13.960147 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.955998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.23.218.193:39791)
0809 18:54:13.956182 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:13.956187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:13.956197 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:54:13.956246 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:13.956248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:13.956263 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:13.956335 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:13.956525 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.957592 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.959162 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.959825 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.959890 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.959941 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:13.960033 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 18:54:13.960073 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260809 18:54:13.960121 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:13.955759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39791 (local address 127.0.0.1:35562)
0809 18:54:13.956057 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:13.956066 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:13.956107 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:13.956414 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:13.956415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:13.956417 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:13.956452 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.956454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.957699 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.957700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.959074 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:13.959077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:13.959904 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:13.959905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:13.959966 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:13.959970 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:54:13.960016 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 18:54:13.960053 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":30,"thread_start_us":89,"threads_started":1}
I20260809 18:54:13.960752 14270 tablet_service.cc:1467] Processing CreateTablet for tablet 6364129cddfd41189557d44328ff3330 (DEFAULT_TABLE table=test-table [id=8eef05f1bf0c4f009893bac903eee013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:13.961082 14270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6364129cddfd41189557d44328ff3330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:13.963591 14630 tablet_bootstrap.cc:492] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: Bootstrap starting.
I20260809 18:54:13.964185 14491 tablet_service.cc:1467] Processing CreateTablet for tablet 6364129cddfd41189557d44328ff3330 (DEFAULT_TABLE table=test-table [id=8eef05f1bf0c4f009893bac903eee013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:13.964376 14630 tablet_bootstrap.cc:654] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:13.964486 14491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6364129cddfd41189557d44328ff3330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:13.964433 14050 tablet_service.cc:1467] Processing CreateTablet for tablet 6364129cddfd41189557d44328ff3330 (DEFAULT_TABLE table=test-table [id=8eef05f1bf0c4f009893bac903eee013]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:13.964783 14050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6364129cddfd41189557d44328ff3330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:13.964993 14630 log.cc:824] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:13.965785 14630 tablet_bootstrap.cc:492] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: No bootstrap required, opened a new log
I20260809 18:54:13.965868 14630 ts_tablet_manager.cc:1397] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:13.968065 14636 tablet_bootstrap.cc:492] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Bootstrap starting.
I20260809 18:54:13.967916 14630 raft_consensus.cc:348] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:13.968142 14630 raft_consensus.cc:374] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:13.968190 14630 raft_consensus.cc:729] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 723df2676ee847a9b8ca07b399413359, State: Initialized, Role: FOLLOWER
I20260809 18:54:13.968302 14630 consensus_queue.cc:260] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:13.968555 14630 ts_tablet_manager.cc:1428] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 18:54:13.968605 14375 heartbeater.cc:503] Master 127.23.218.254:38173 was elected leader, sending a full tablet report...
I20260809 18:54:13.968778 14636 tablet_bootstrap.cc:654] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:13.968777 14633 tablet_bootstrap.cc:492] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Bootstrap starting.
I20260809 18:54:13.969219 14636 log.cc:824] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:13.969410 14633 tablet_bootstrap.cc:654] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:13.969934 14633 log.cc:824] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:13.970017 14636 tablet_bootstrap.cc:492] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: No bootstrap required, opened a new log
I20260809 18:54:13.970084 14636 ts_tablet_manager.cc:1397] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:54:13.970570 14633 tablet_bootstrap.cc:492] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: No bootstrap required, opened a new log
I20260809 18:54:13.970624 14633 ts_tablet_manager.cc:1397] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:54:13.971623 14636 raft_consensus.cc:348] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:13.971740 14636 raft_consensus.cc:374] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:13.971761 14636 raft_consensus.cc:729] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b4a8593d9740aa9b3100d2db8b979c, State: Initialized, Role: FOLLOWER
I20260809 18:54:13.971850 14636 consensus_queue.cc:260] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:13.972066 14636 ts_tablet_manager.cc:1428] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:13.972114 14165 heartbeater.cc:503] Master 127.23.218.254:38173 was elected leader, sending a full tablet report...
I20260809 18:54:13.971995 14633 raft_consensus.cc:348] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:13.972401 14633 raft_consensus.cc:374] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:13.972455 14633 raft_consensus.cc:729] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e0b19e67d448728b16a8146c33ada6, State: Initialized, Role: FOLLOWER
I20260809 18:54:13.972561 14633 consensus_queue.cc:260] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:13.972832 14633 ts_tablet_manager.cc:1428] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 18:54:13.972846 14607 heartbeater.cc:503] Master 127.23.218.254:38173 was elected leader, sending a full tablet report...
W20260809 18:54:14.026816 14377 tablet.cc:2369] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:54:14.123142 14168 tablet.cc:2369] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:54:14.177899 14611 tablet.cc:2369] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:54:14.237401 14642 raft_consensus.cc:482] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:14.237507 14642 raft_consensus.cc:504] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.237830 14642 leader_election.cc:288] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725), 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193:39791)
I20260809 18:54:14.239866 14641 raft_consensus.cc:482] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:14.239966 14641 raft_consensus.cc:504] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.240614 14641 leader_election.cc:288] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939), 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725)
I20260809 18:54:14.241585 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.238061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:44029 (local address 127.23.218.193:39791)
0809 18:54:14.238109 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.238111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.238223 (+ 112us) server_negotiation.cc:407] Connection header received
0809 18:54:14.238262 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.238263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.238272 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.238317 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:14.238474 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.239201 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.240701 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.241243 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.241282 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.241291 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.241483 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.241520 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 18:54:14.241611 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.237978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:38725 (local address 127.23.218.195:45319)
0809 18:54:14.238045 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.238057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.238095 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.238257 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.238258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.238260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:14.238364 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.238366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.239280 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.239280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.240594 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.240599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.241380 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.241381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.241470 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.241474 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.241501 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.241541 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":62}
I20260809 18:54:14.241925 14625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.238053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:45319 (local address 127.23.218.194:38725)
0809 18:54:14.238099 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.238102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.238148 (+ 46us) server_negotiation.cc:407] Connection header received
0809 18:54:14.238174 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.238174 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.238183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.238226 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:14.238432 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.239202 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.240741 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.241289 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.241803 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.241812 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.241828 (+ 16us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.241863 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 18:54:14.242004 14643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.237990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39791 (local address 127.23.218.195:44029)
0809 18:54:14.238162 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.238170 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.238219 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.238332 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.238332 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.238333 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:14.238407 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.238408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.239275 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.239276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.240595 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.240599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.241342 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.241342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.241414 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.241418 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.241885 (+ 467us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.241929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":63,"thread_start_us":106,"threads_started":1}
I20260809 18:54:14.242414 14301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6364129cddfd41189557d44328ff3330" candidate_uuid: "57e0b19e67d448728b16a8146c33ada6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723df2676ee847a9b8ca07b399413359" is_pre_election: true
I20260809 18:54:14.242419 14085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6364129cddfd41189557d44328ff3330" candidate_uuid: "57e0b19e67d448728b16a8146c33ada6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b4a8593d9740aa9b3100d2db8b979c" is_pre_election: true
I20260809 18:54:14.242522 14085 raft_consensus.cc:2454] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e0b19e67d448728b16a8146c33ada6 in term 0.
I20260809 18:54:14.242556 14301 raft_consensus.cc:2454] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e0b19e67d448728b16a8146c33ada6 in term 0.
I20260809 18:54:14.242698 14416 leader_election.cc:302] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20b4a8593d9740aa9b3100d2db8b979c, 57e0b19e67d448728b16a8146c33ada6; no voters:
I20260809 18:54:14.242905 14642 raft_consensus.cc:2781] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:54:14.242995 14642 raft_consensus.cc:482] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:54:14.243036 14642 raft_consensus.cc:3037] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:14.243790 14624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.240528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:60179 (local address 127.23.218.195:44939)
0809 18:54:14.240566 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.240568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.240895 (+ 327us) server_negotiation.cc:407] Connection header received
0809 18:54:14.240970 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.240971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.240979 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.241003 (+ 24us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:14.241227 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.241859 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.243141 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.243488 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.243550 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.243556 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.243697 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.243733 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:54:14.243848 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.240378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:38725 (local address 127.23.218.193:53015)
0809 18:54:14.240706 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.240716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.240769 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.240926 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.240928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.240930 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:14.240990 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.240991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.241835 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.241836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.242940 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.242943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.243620 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.243621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.243679 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.243681 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.243734 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.243762 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":47,"threads_started":1}
I20260809 18:54:14.243781 14644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.240378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:44939 (local address 127.23.218.193:60179)
0809 18:54:14.240849 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.240857 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.240914 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.241063 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.241063 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.241078 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:14.241144 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.241146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.241949 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.241949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.243044 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.243047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.243560 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.243561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.243625 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.243629 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.243673 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.243706 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":15,"thread_start_us":38,"threads_started":1}
I20260809 18:54:14.243919 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.240513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:53015 (local address 127.23.218.194:38725)
0809 18:54:14.240635 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.240637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.240759 (+ 122us) server_negotiation.cc:407] Connection header received
0809 18:54:14.240811 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.240811 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.240821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.241044 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:14.241095 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.241747 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.243033 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.243552 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.243625 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.243634 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.243814 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.243855 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260809 18:54:14.244052 14301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6364129cddfd41189557d44328ff3330" candidate_uuid: "20b4a8593d9740aa9b3100d2db8b979c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723df2676ee847a9b8ca07b399413359" is_pre_election: true
I20260809 18:54:14.244113 14301 raft_consensus.cc:2454] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20b4a8593d9740aa9b3100d2db8b979c in term 0.
I20260809 18:54:14.244108 14642 raft_consensus.cc:504] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.244297 13966 leader_election.cc:302] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20b4a8593d9740aa9b3100d2db8b979c, 723df2676ee847a9b8ca07b399413359; no voters:
I20260809 18:54:14.244279 14525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6364129cddfd41189557d44328ff3330" candidate_uuid: "20b4a8593d9740aa9b3100d2db8b979c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e0b19e67d448728b16a8146c33ada6" is_pre_election: true
I20260809 18:54:14.244333 14301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6364129cddfd41189557d44328ff3330" candidate_uuid: "57e0b19e67d448728b16a8146c33ada6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723df2676ee847a9b8ca07b399413359"
I20260809 18:54:14.244380 14525 raft_consensus.cc:2379] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20b4a8593d9740aa9b3100d2db8b979c in current term 1: Already voted for candidate 57e0b19e67d448728b16a8146c33ada6 in this term.
I20260809 18:54:14.244395 14301 raft_consensus.cc:3037] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:14.245142 14301 raft_consensus.cc:2454] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e0b19e67d448728b16a8146c33ada6 in term 1.
I20260809 18:54:14.245237 14642 leader_election.cc:288] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [CANDIDATE]: Term 1 election: Requested vote from peers 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725), 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193:39791)
I20260809 18:54:14.245297 14413 leader_election.cc:302] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e0b19e67d448728b16a8146c33ada6, 723df2676ee847a9b8ca07b399413359; no voters:
I20260809 18:54:14.245379 14642 raft_consensus.cc:2781] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:14.245401 14085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6364129cddfd41189557d44328ff3330" candidate_uuid: "57e0b19e67d448728b16a8146c33ada6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b4a8593d9740aa9b3100d2db8b979c"
I20260809 18:54:14.245530 14642 raft_consensus.cc:686] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 1 LEADER]: Becoming Leader. State: Replica: 57e0b19e67d448728b16a8146c33ada6, State: Running, Role: LEADER
I20260809 18:54:14.245617 14642 consensus_queue.cc:237] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.245343 14641 raft_consensus.cc:2781] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:54:14.246043 14085 raft_consensus.cc:3037] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:14.246311 13851 catalog_manager.cc:5495] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195). New cstate: current_term: 1 leader_uuid: "57e0b19e67d448728b16a8146c33ada6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } health_report { overall_health: UNKNOWN } } }
I20260809 18:54:14.246846 14085 raft_consensus.cc:2454] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e0b19e67d448728b16a8146c33ada6 in term 1.
I20260809 18:54:14.246999 14641 raft_consensus.cc:482] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:54:14.247035 14641 raft_consensus.cc:3037] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Advancing to term 2
I20260809 18:54:14.247577 14641 raft_consensus.cc:504] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.247802 14525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6364129cddfd41189557d44328ff3330" candidate_uuid: "20b4a8593d9740aa9b3100d2db8b979c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57e0b19e67d448728b16a8146c33ada6"
I20260809 18:54:14.247804 14301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6364129cddfd41189557d44328ff3330" candidate_uuid: "20b4a8593d9740aa9b3100d2db8b979c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723df2676ee847a9b8ca07b399413359"
I20260809 18:54:14.247871 14301 raft_consensus.cc:3037] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 1 FOLLOWER]: Advancing to term 2
I20260809 18:54:14.248324 14301 raft_consensus.cc:2454] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20b4a8593d9740aa9b3100d2db8b979c in term 2.
I20260809 18:54:14.247754 14641 leader_election.cc:288] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 election: Requested vote from peers 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939), 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725)
I20260809 18:54:14.248461 13966 leader_election.cc:302] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20b4a8593d9740aa9b3100d2db8b979c, 723df2676ee847a9b8ca07b399413359; no voters: 57e0b19e67d448728b16a8146c33ada6
I20260809 18:54:14.248534 14641 raft_consensus.cc:2781] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 2 FOLLOWER]: Leader election won for term 2
I20260809 18:54:14.248663 14641 raft_consensus.cc:686] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 2 LEADER]: Becoming Leader. State: Replica: 20b4a8593d9740aa9b3100d2db8b979c, State: Running, Role: LEADER
I20260809 18:54:14.248757 14641 consensus_queue.cc:237] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.249380 13851 catalog_manager.cc:5495] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c reported cstate change: term changed from 1 to 2, leader changed from 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195) to 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193). New cstate: current_term: 2 leader_uuid: "20b4a8593d9740aa9b3100d2db8b979c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } health_report { overall_health: HEALTHY } } }
I20260809 18:54:14.308336 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.306131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35572 (local address 127.23.218.193:39791)
0809 18:54:14.306177 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.306179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.306187 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:54:14.306274 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.306275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.306292 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.306383 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:54:14.306490 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.306926 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.307647 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.307831 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.307884 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.308034 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:54:14.308127 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:54:14.308196 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.308256 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.308274 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 18:54:14.309783 14525 raft_consensus.cc:3032] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 18:54:14.309835 14525 raft_consensus.cc:729] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 57e0b19e67d448728b16a8146c33ada6, State: Running, Role: LEADER
I20260809 18:54:14.309844 14301 raft_consensus.cc:1261] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 2 FOLLOWER]: Refusing update from remote peer 20b4a8593d9740aa9b3100d2db8b979c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 18:54:14.309876 14525 consensus_queue.cc:260] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.309963 14525 raft_consensus.cc:3037] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 1 FOLLOWER]: Advancing to term 2
I20260809 18:54:14.310112 14655 consensus_queue.cc:1035] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [LEADER]: Connected to new peer: Peer: permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:14.310699 14525 raft_consensus.cc:1261] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 2 FOLLOWER]: Refusing update from remote peer 20b4a8593d9740aa9b3100d2db8b979c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 18:54:14.310920 14655 consensus_queue.cc:1035] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [LEADER]: Connected to new peer: Peer: permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:14.311000 14655 consensus_queue.cc:1225] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [LEADER]: Peer 57e0b19e67d448728b16a8146c33ada6 log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260809 18:54:14.311288 14525 pending_rounds.cc:76] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Aborting all ops after (but not including) 0
I20260809 18:54:14.311335 14525 pending_rounds.cc:97] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260809 18:54:14.311384 14525 raft_consensus.cc:2866] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 18:54:14.312049 14659 mvcc.cc:205] Tried to move back new op lower bound from 7316691576050937856 to 7316691575803351040. Current Snapshot: MvccSnapshot[applied={T|T < 7316691576050937856}]
I20260809 18:54:14.312764 14661 mvcc.cc:205] Tried to move back new op lower bound from 7316691576050937856 to 7316691575803351040. Current Snapshot: MvccSnapshot[applied={T|T < 7316691576050937856}]
I20260809 18:54:14.313153 14665 mvcc.cc:205] Tried to move back new op lower bound from 7316691576050937856 to 7316691575803351040. Current Snapshot: MvccSnapshot[applied={T|T < 7316691576050937856 or (T in {7316691576050937856})}]
I20260809 18:54:14.314566 13847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:14.314648 13847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:14.315387 13847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Sending DeleteTablet for 3 replicas of tablet 6364129cddfd41189557d44328ff3330
I20260809 18:54:14.315752 14270 tablet_service.cc:1514] Processing DeleteTablet for tablet 6364129cddfd41189557d44328ff3330 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:14 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:34746
I20260809 18:54:14.315907 14491 tablet_service.cc:1514] Processing DeleteTablet for tablet 6364129cddfd41189557d44328ff3330 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:14 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:43190
I20260809 18:54:14.315985 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 13942
I20260809 18:54:14.315986 14671 tablet_replica.cc:330] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: stopping tablet replica
I20260809 18:54:14.316079 14671 raft_consensus.cc:2229] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:14.316201 14671 raft_consensus.cc:2258] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:14.324862 14671 ts_tablet_manager.cc:1905] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:14.325222 14672 tablet_replica.cc:330] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: stopping tablet replica
I20260809 18:54:14.325363 14672 raft_consensus.cc:2229] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:14.325484 14672 raft_consensus.cc:2258] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:14.325942 14672 ts_tablet_manager.cc:1905] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:14.326314 14671 ts_tablet_manager.cc:1918] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260809 18:54:14.326409 14671 log.cc:1196] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/6364129cddfd41189557d44328ff3330
I20260809 18:54:14.326640 14671 ts_tablet_manager.cc:1939] T 6364129cddfd41189557d44328ff3330 P 723df2676ee847a9b8ca07b399413359: Deleting consensus metadata
I20260809 18:54:14.327013 13820 catalog_manager.cc:4841] TS 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725): tablet 6364129cddfd41189557d44328ff3330 (table test-table [id=8eef05f1bf0c4f009893bac903eee013]) successfully deleted
I20260809 18:54:14.327360 14672 ts_tablet_manager.cc:1918] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260809 18:54:14.327435 14672 log.cc:1196] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/6364129cddfd41189557d44328ff3330
I20260809 18:54:14.327688 14672 ts_tablet_manager.cc:1939] T 6364129cddfd41189557d44328ff3330 P 57e0b19e67d448728b16a8146c33ada6: Deleting consensus metadata
I20260809 18:54:14.328055 13826 catalog_manager.cc:4841] TS 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939): tablet 6364129cddfd41189557d44328ff3330 (table test-table [id=8eef05f1bf0c4f009893bac903eee013]) successfully deleted
W20260809 18:54:14.329784 13826 connection.cc:535] client connection to 127.23.218.193:39791 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 18:54:14.329877 13826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 18:54:14.330274 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.330156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39791 (local address 127.0.0.1:35584)
0809 18:54:14.330214 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.330250 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:39791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 18:54:14.330277 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 14174
W20260809 18:54:14.341043 13826 catalog_manager.cc:4584] TS 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193:39791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6364129cddfd41189557d44328ff3330: Network error: Client connection negotiation failed: client connection to 127.23.218.193:39791: connect: Connection refused (error 111)
I20260809 18:54:14.344499 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 14389
I20260809 18:54:14.360170 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 13794
I20260809 18:54:14.367498 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38173
--webserver_interface=127.23.218.254
--webserver_port=35487
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:14.444571 14674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:14.444739 14674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:14.444767 14674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:14.445912 14674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:14.445951 14674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:14.445971 14674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:14.445984 14674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:14.446000 14674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:14.447440 14674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=35487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:14.447650 14674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:14.447844 14674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:14.486084 14686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:14.486095 14682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:14.486073 14681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:14.486259 14674 server_base.cc:1034] running on GCE node
I20260809 18:54:14.486652 14674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:14.486922 14674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:14.488049 14674 hybrid_clock.cc:648] HybridClock initialized: now 1786301654488036 us; error 30 us; skew 500 ppm
Aug 09 18:54:14 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301654, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:14.490413 14674 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:54:14.490837 14674 webserver.cc:460] Webserver started at http://127.23.218.254:35487/ using document root <none> and password file <none>
I20260809 18:54:14.491034 14674 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:14.491091 14674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:14.492329 14674 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:54:14.492883 14696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:14.493022 14674 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:54:14.493104 14674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "a0b3e4d6a65f49299690e11d502377e8"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "1e06275f14aac70f319854cbfd53b907"
server_key_iv: "d1d10ee465fbd3ebfb7c7e4acd6ce8ec"
server_key_version: "encryptionkey@0"
I20260809 18:54:14.493460 14674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:14.507149 14674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:14.508664 14674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:14.508783 14674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:14.512527 14674 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:38173
I20260809 18:54:14.512570 14788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:38173 every 8 connection(s)
I20260809 18:54:14.512897 14674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:54:14.514328 14789 sys_catalog.cc:263] Verifying existing consensus state
I20260809 18:54:14.514995 14789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Bootstrap starting.
I20260809 18:54:14.522136 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 14674
I20260809 18:54:14.522423 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:39791
--local_ip_for_outbound_sockets=127.23.218.193
--tserver_master_addrs=127.23.218.254:38173
--webserver_port=44619
--webserver_interface=127.23.218.193
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:54:14.523561 14789 log.cc:824] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:14.525239 14789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=5 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:54:14.525477 14789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Bootstrap complete.
I20260809 18:54:14.527529 14789 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:14.527740 14789 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b3e4d6a65f49299690e11d502377e8, State: Initialized, Role: FOLLOWER
I20260809 18:54:14.527887 14789 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:14.527959 14789 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:54:14.528012 14789 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:54:14.528059 14789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 1 FOLLOWER]: Advancing to term 2
I20260809 18:54:14.528971 14789 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:14.529069 14789 leader_election.cc:302] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0b3e4d6a65f49299690e11d502377e8; no voters:
I20260809 18:54:14.529222 14789 leader_election.cc:288] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 18:54:14.529280 14799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 18:54:14.529448 14789 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:14.529465 14799 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 2 LEADER]: Becoming Leader. State: Replica: a0b3e4d6a65f49299690e11d502377e8, State: Running, Role: LEADER
I20260809 18:54:14.529663 14799 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:14.529966 14802 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a0b3e4d6a65f49299690e11d502377e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } } }
I20260809 18:54:14.530052 14802 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:14.530259 14803 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0b3e4d6a65f49299690e11d502377e8. Latest consensus state: current_term: 2 leader_uuid: "a0b3e4d6a65f49299690e11d502377e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } } }
I20260809 18:54:14.530328 14803 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:14.530332 14812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:54:14.531231 14812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:54:14.531384 14812 catalog_manager.cc:1259] Loaded cluster ID: c73d8d5399a14430850eea3b74832051
I20260809 18:54:14.531605 14812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:54:14.531862 14812 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 18:54:14.536463 14812 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 18:54:14.538744 14812 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:54:14.538959 14812 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Loaded TSK: 0
I20260809 18:54:14.539178 14812 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 18:54:14.603667 14795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:14.603912 14795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:14.603946 14795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:14.605551 14795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:14.605602 14795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:14.605635 14795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:54:14.607692 14795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:39791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=44619
--tserver_master_addrs=127.23.218.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:14.607941 14795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:14.608192 14795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:14.613616 14844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:14.613783 14849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:14.613884 14845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:14.614495 14795 server_base.cc:1034] running on GCE node
I20260809 18:54:14.614717 14795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:14.614946 14795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:14.616099 14795 hybrid_clock.cc:648] HybridClock initialized: now 1786301654616078 us; error 39 us; skew 500 ppm
Aug 09 18:54:14 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301654, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:14.618594 14795 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:54:14.619041 14795 webserver.cc:460] Webserver started at http://127.23.218.193:44619/ using document root <none> and password file <none>
I20260809 18:54:14.619253 14795 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:14.619314 14795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:14.620594 14795 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:14.621227 14857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:14.621405 14795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:14.621505 14795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "20b4a8593d9740aa9b3100d2db8b979c"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "186f4908c61ed6e95f7a584fd3f261e9"
server_key_iv: "85dbedbc03dd7cc12fd3249d5c3cbfaf"
server_key_version: "encryptionkey@0"
I20260809 18:54:14.621910 14795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:14.641835 14795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:14.643987 14795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:14.644136 14795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:14.644356 14795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:14.644832 14867 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 18:54:14.646255 14795 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 18:54:14.646311 14795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 18:54:14.646360 14795 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 18:54:14.647137 14795 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 18:54:14.647176 14795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 18:54:14.647249 14867 tablet_bootstrap.cc:492] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Bootstrap starting.
I20260809 18:54:14.653638 14795 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:39791
I20260809 18:54:14.653865 15044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:39791 every 8 connection(s)
I20260809 18:54:14.654037 14795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:54:14.657478 14867 log.cc:824] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:14.658092 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 14795
I20260809 18:54:14.658401 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:38725
--local_ip_for_outbound_sockets=127.23.218.194
--tserver_master_addrs=127.23.218.254:38173
--webserver_port=39091
--webserver_interface=127.23.218.194
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:54:14.662055 14867 tablet_bootstrap.cc:492] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:54:14.662355 14867 tablet_bootstrap.cc:492] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Bootstrap complete.
I20260809 18:54:14.663472 14867 ts_tablet_manager.cc:1397] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.012s
Aug 09 18:54:14 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301654, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:14.664754 14867 raft_consensus.cc:348] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.665064 14867 raft_consensus.cc:729] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b4a8593d9740aa9b3100d2db8b979c, State: Initialized, Role: FOLLOWER
I20260809 18:54:14.665210 14867 consensus_queue.cc:260] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.665484 14867 ts_tablet_manager.cc:1428] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 18:54:14.667799 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.654755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:48909)
0809 18:54:14.655037 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.655077 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.655682 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.658294 (+ 2612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.658299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.658638 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:14.658871 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.658877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.660379 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.660380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.661571 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.661574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.662370 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.662372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.662506 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.663010 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:14.663019 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:14.664590 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:14.666327 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:14.666333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:14.666355 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:14.666564 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:14.666791 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:14.666792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:14.666793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:14.666892 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:14.667238 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:14.667245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:14.667364 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.667440 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.667608 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":98,"threads_started":1}
I20260809 18:54:14.667799 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.654821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:48909 (local address 127.23.218.254:38173)
0809 18:54:14.657804 (+ 2983us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.657813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.657850 (+ 37us) server_negotiation.cc:407] Connection header received
0809 18:54:14.657983 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.657990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.658047 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.658207 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:14.659003 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.660242 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.661673 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.662223 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.662756 (+ 533us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.664699 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:14.664703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:14.664703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:14.664714 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:14.666250 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.666640 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.666642 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.666643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.666732 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.666958 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.666959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.666959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.667171 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:14.667239 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.667519 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.667638 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2931,"thread_start_us":79,"threads_started":1}
I20260809 18:54:14.668479 15045 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38173
I20260809 18:54:14.668579 15045 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:14.668773 15045 heartbeater.cc:511] Master 127.23.218.254:38173 requested a full tablet report, sending...
I20260809 18:54:14.669524 14731 ts_manager.cc:194] Registered new tserver with Master: 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193:39791)
I20260809 18:54:14.670749 14731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:48909
I20260809 18:54:14.671623 15045 heartbeater.cc:503] Master 127.23.218.254:38173 was elected leader, sending a full tablet report...
Aug 09 18:54:14 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301654, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:54:14.677037 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.670179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.23.218.193:39791)
0809 18:54:14.670603 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.670607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.670613 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:54:14.670636 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.670637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.670647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.670699 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:14.671142 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.672042 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.672723 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.672998 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.673052 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.674743 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:14.674750 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:14.674751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:14.674757 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:14.675983 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.676311 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.676314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.676314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.676367 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.676526 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.676527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.676527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.676653 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:14.676721 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.676880 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.676957 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":52,"threads_started":1}
I20260809 18:54:14.677037 15058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.670140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39791 (local address 127.0.0.1:35590)
0809 18:54:14.670433 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.670459 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.670507 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.670717 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.670723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.670988 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:14.671041 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.671044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.672110 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.672111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.672650 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.672652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.673061 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.673062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.673129 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.673505 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:14.673509 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:14.674666 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:14.676056 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:14.676062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:14.676064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:14.676240 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:14.676420 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:14.676421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:14.676421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:14.676476 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:14.676712 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:14.676719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:14.676768 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.676839 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.676946 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":84,"threads_started":1}
I20260809 18:54:14.677426 14904 tablet_service.cc:1514] Processing DeleteTablet for tablet 6364129cddfd41189557d44328ff3330 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:14 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:35590
I20260809 18:54:14.677845 15061 tablet_replica.cc:330] stopping tablet replica
I20260809 18:54:14.677945 15061 raft_consensus.cc:2229] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:14.678011 15061 raft_consensus.cc:2258] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:14.678314 15061 ts_tablet_manager.cc:1905] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:14.680305 15061 ts_tablet_manager.cc:1918] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260809 18:54:14.680378 15061 log.cc:1196] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/6364129cddfd41189557d44328ff3330
I20260809 18:54:14.680588 15061 ts_tablet_manager.cc:1939] T 6364129cddfd41189557d44328ff3330 P 20b4a8593d9740aa9b3100d2db8b979c: Deleting consensus metadata
I20260809 18:54:14.681044 14698 catalog_manager.cc:4841] TS 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193:39791): tablet 6364129cddfd41189557d44328ff3330 (table test-table [id=8eef05f1bf0c4f009893bac903eee013]) successfully deleted
W20260809 18:54:14.741958 15053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:14.742138 15053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:14.742159 15053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:14.743237 15053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:14.743275 15053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:14.743299 15053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:54:14.744658 15053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:38725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=39091
--tserver_master_addrs=127.23.218.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:14.744851 15053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:14.745013 15053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:14.749446 15078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:14.749459 15072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:14.749447 15068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:14.749647 15053 server_base.cc:1034] running on GCE node
I20260809 18:54:14.749823 15053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:14.750041 15053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:14.751173 15053 hybrid_clock.cc:648] HybridClock initialized: now 1786301654751160 us; error 27 us; skew 500 ppm
Aug 09 18:54:14 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301654, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:14.753403 15053 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:54:14.753772 15053 webserver.cc:460] Webserver started at http://127.23.218.194:39091/ using document root <none> and password file <none>
I20260809 18:54:14.753968 15053 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:14.754024 15053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:14.755327 15053 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:14.755834 15090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:14.755991 15053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:14.756069 15053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "723df2676ee847a9b8ca07b399413359"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "ed8eba22cbec483cd1dd0a1ba5a7a666"
server_key_iv: "fe06cb09640780345a0f7bdf8e8dfcc0"
server_key_version: "encryptionkey@0"
I20260809 18:54:14.756359 15053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:14.777285 15053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:14.779199 15053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:14.779330 15053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:14.779510 15053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:14.779762 15053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:14.779794 15053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:14.779820 15053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:14.779837 15053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:14.785151 15053 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:38725
I20260809 18:54:14.785204 15275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:38725 every 8 connection(s)
I20260809 18:54:14.785496 15053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
Aug 09 18:54:14 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301654, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:14.793375 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.786083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:57339 (local address 127.23.218.254:38173)
0809 18:54:14.786123 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.786125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.786302 (+ 177us) server_negotiation.cc:407] Connection header received
0809 18:54:14.786785 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.786786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.786802 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.786851 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:14.787714 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.788200 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.789117 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.789334 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.789389 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.791231 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:14.791233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:14.791233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:14.791240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:14.792411 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.792725 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.792726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.792726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.792770 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.792932 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.792933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.792933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.793022 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:14.793063 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.793252 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.793314 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:54:14.793493 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.786065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:57339)
0809 18:54:14.786196 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.786230 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.786685 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.786914 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.786918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.787451 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:14.787620 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.787624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.788287 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.788288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.789038 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.789040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.789403 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.789404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.789516 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.789908 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:14.789916 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:14.791138 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:14.792476 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:14.792482 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:14.792483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:14.792672 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:14.792812 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:14.792812 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:14.792813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:14.792892 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:14.793061 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:14.793063 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:14.793158 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.793236 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.793339 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260809 18:54:14.793924 15276 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38173
I20260809 18:54:14.794008 15276 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:14.794160 15276 heartbeater.cc:511] Master 127.23.218.254:38173 requested a full tablet report, sending...
I20260809 18:54:14.794548 14731 ts_manager.cc:194] Registered new tserver with Master: 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725)
I20260809 18:54:14.794878 14731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:57339
I20260809 18:54:14.795449 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 15053
I20260809 18:54:14.795701 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:44939
--local_ip_for_outbound_sockets=127.23.218.195
--tserver_master_addrs=127.23.218.254:38173
--webserver_port=40211
--webserver_interface=127.23.218.195
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:14.875234 15286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:14.875404 15286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:14.875435 15286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:14.876513 15286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:14.876549 15286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:14.876585 15286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:54:14.877959 15286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:44939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=40211
--tserver_master_addrs=127.23.218.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:14.878159 15286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:14.878360 15286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:14.882860 15297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:14.882820 15300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:14.882836 15298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:14.882939 15286 server_base.cc:1034] running on GCE node
I20260809 18:54:14.883229 15286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:14.883435 15286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:14.884575 15286 hybrid_clock.cc:648] HybridClock initialized: now 1786301654884561 us; error 27 us; skew 500 ppm
Aug 09 18:54:14 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301654, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:14.886788 15286 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:54:14.887123 15286 webserver.cc:460] Webserver started at http://127.23.218.195:40211/ using document root <none> and password file <none>
I20260809 18:54:14.887275 15286 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:14.887324 15286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:14.888437 15286 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:14.888903 15309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:14.889037 15286 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:14.889112 15286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "57e0b19e67d448728b16a8146c33ada6"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "30538104bd5fdcf0c9337f56c64b9efd"
server_key_iv: "ee945a8a6611f0be433df1db59192aa2"
server_key_version: "encryptionkey@0"
I20260809 18:54:14.889375 15286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:14.899756 15286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:14.901566 15286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:14.901664 15286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:14.901827 15286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:14.902036 15286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:14.902063 15286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:14.902081 15286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:14.902092 15286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:14.907320 15286 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:44939
I20260809 18:54:14.907356 15488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:44939 every 8 connection(s)
I20260809 18:54:14.907619 15286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:54:14.910450 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 15286
Aug 09 18:54:14 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301654, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:14.916561 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.908211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:41061 (local address 127.23.218.254:38173)
0809 18:54:14.908256 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.908258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.908422 (+ 164us) server_negotiation.cc:407] Connection header received
0809 18:54:14.908858 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.908859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.908872 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.908921 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:14.909742 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.910222 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.911492 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.911701 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.911761 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.913856 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:14.913857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:14.913857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:14.913865 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:14.915301 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.915707 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.915708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.915708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.915764 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.915959 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.915960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.915960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.916082 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:14.916145 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.916404 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.916478 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:54:14.916709 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.908188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:41061)
0809 18:54:14.908320 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.908352 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.908763 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.908980 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.908984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.909484 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:14.909647 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.909652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.910315 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.910316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.911416 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.911419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.911782 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.911783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.911879 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.912255 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:14.912264 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:14.913775 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:14.915369 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:14.915375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:14.915377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:14.915642 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:14.915821 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:14.915821 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:14.915822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:14.915909 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:14.916137 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:14.916140 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:14.916279 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.916386 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.916513 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260809 18:54:14.917124 15491 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38173
I20260809 18:54:14.917208 15491 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:14.917287 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.910777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.23.218.254:38173)
0809 18:54:14.910867 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.910869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.910911 (+ 42us) server_negotiation.cc:407] Connection header received
0809 18:54:14.910981 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.910982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.910990 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.911039 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:14.911649 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.912224 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.913139 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.913442 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.913493 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.914707 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:14.914708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:14.914709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:14.914716 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:14.916081 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.916438 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.916439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.916439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.916492 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:14.916797 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:14.916798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:14.916798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:14.916924 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:14.917075 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.917129 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.917202 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":50,"threads_started":1}
I20260809 18:54:14.917374 15491 heartbeater.cc:511] Master 127.23.218.254:38173 requested a full tablet report, sending...
I20260809 18:54:14.917730 14731 ts_manager.cc:194] Registered new tserver with Master: 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939)
I20260809 18:54:14.918076 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:54:14.918222 14731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:41061
I20260809 18:54:14.920565 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.918386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.23.218.254:38173)
0809 18:54:14.918531 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.918533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.918539 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:54:14.918569 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.918570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.918606 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.918659 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:54:14.918776 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.919212 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.919889 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.920088 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.920147 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.920294 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:54:14.920382 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:54:14.920445 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.920481 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.920504 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260809 18:54:14.920800 14731 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:54:14.922284 14731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:54:14.924209 14904 tablet_service.cc:1467] Processing CreateTablet for tablet 1c04f52d45344a1891537e2e8a990047 (DEFAULT_TABLE table=test-table [id=cc06df3055274231879b31eff59a06c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:14.924420 14904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c04f52d45344a1891537e2e8a990047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:14.925729 14867 tablet_bootstrap.cc:492] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Bootstrap starting.
I20260809 18:54:14.926026 14867 tablet_bootstrap.cc:654] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:14.927011 14867 tablet_bootstrap.cc:492] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: No bootstrap required, opened a new log
I20260809 18:54:14.927063 14867 ts_tablet_manager.cc:1397] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:14.927215 14867 raft_consensus.cc:348] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.927302 14867 raft_consensus.cc:374] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:14.927356 14867 raft_consensus.cc:729] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b4a8593d9740aa9b3100d2db8b979c, State: Initialized, Role: FOLLOWER
I20260809 18:54:14.927400 14867 consensus_queue.cc:260] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.927485 14867 ts_tablet_manager.cc:1428] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:14.928406 15501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.924094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.23.218.195:44939)
0809 18:54:14.924251 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.924256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.924265 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:54:14.924297 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.924299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.924314 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.924392 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:14.924525 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.925616 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.927422 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.928057 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.928140 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.928193 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.928300 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.928334 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260809 18:54:14.928397 15058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.923893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:44939 (local address 127.0.0.1:43202)
0809 18:54:14.923991 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.924001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.924029 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.924399 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.924400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.924402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:14.924447 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.924448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.926420 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.926421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.927336 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.927339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.928157 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.928157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.928211 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.928237 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.928277 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.928322 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 18:54:14.928406 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.923914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:38725 (local address 127.0.0.1:34760)
0809 18:54:14.924247 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.924255 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.924305 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.924588 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.924589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.924591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:14.924633 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.924634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.926061 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.926062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.927148 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.927151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.928157 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.928158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.928234 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.928237 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.928289 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.928329 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":11,"thread_start_us":81,"threads_started":1}
I20260809 18:54:14.928434 15500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.924021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34760 (local address 127.23.218.194:38725)
0809 18:54:14.924190 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.924195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.924386 (+ 191us) server_negotiation.cc:407] Connection header received
0809 18:54:14.924433 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.924435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.924451 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.924519 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:14.924707 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.925975 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.927232 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.927952 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.928020 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.928074 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.928318 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.928362 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260809 18:54:14.929118 15392 tablet_service.cc:1467] Processing CreateTablet for tablet 1c04f52d45344a1891537e2e8a990047 (DEFAULT_TABLE table=test-table [id=cc06df3055274231879b31eff59a06c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:14.929096 15164 tablet_service.cc:1467] Processing CreateTablet for tablet 1c04f52d45344a1891537e2e8a990047 (DEFAULT_TABLE table=test-table [id=cc06df3055274231879b31eff59a06c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:14.929445 15164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c04f52d45344a1891537e2e8a990047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:14.929445 15392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c04f52d45344a1891537e2e8a990047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:14.932713 15507 tablet_bootstrap.cc:492] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: Bootstrap starting.
I20260809 18:54:14.933346 15507 tablet_bootstrap.cc:654] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:14.933741 15507 log.cc:824] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:14.934017 15509 tablet_bootstrap.cc:492] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: Bootstrap starting.
I20260809 18:54:14.934379 15507 tablet_bootstrap.cc:492] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: No bootstrap required, opened a new log
I20260809 18:54:14.934439 15507 ts_tablet_manager.cc:1397] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:54:14.934572 15509 tablet_bootstrap.cc:654] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:14.934969 15509 log.cc:824] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:14.935585 15509 tablet_bootstrap.cc:492] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: No bootstrap required, opened a new log
I20260809 18:54:14.935642 15509 ts_tablet_manager.cc:1397] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:54:14.935925 15507 raft_consensus.cc:348] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.936041 15507 raft_consensus.cc:374] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:14.936060 15507 raft_consensus.cc:729] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57e0b19e67d448728b16a8146c33ada6, State: Initialized, Role: FOLLOWER
I20260809 18:54:14.936163 15507 consensus_queue.cc:260] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.936426 15491 heartbeater.cc:503] Master 127.23.218.254:38173 was elected leader, sending a full tablet report...
I20260809 18:54:14.936412 15507 ts_tablet_manager.cc:1428] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 18:54:14.937237 15509 raft_consensus.cc:348] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.937390 15509 raft_consensus.cc:374] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:14.937429 15509 raft_consensus.cc:729] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 723df2676ee847a9b8ca07b399413359, State: Initialized, Role: FOLLOWER
I20260809 18:54:14.937532 15509 consensus_queue.cc:260] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.937765 15509 ts_tablet_manager.cc:1428] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:14.937790 15276 heartbeater.cc:503] Master 127.23.218.254:38173 was elected leader, sending a full tablet report...
I20260809 18:54:14.972625 15512 raft_consensus.cc:482] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:14.972713 15512 raft_consensus.cc:504] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.973042 15512 leader_election.cc:288] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725), 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193:39791)
I20260809 18:54:14.975556 15493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.973167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:38725 (local address 127.23.218.195:60317)
0809 18:54:14.973240 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.973249 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.973288 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.973457 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.973458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.973459 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:14.973518 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.973519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.974077 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.974078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.974831 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.974834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.975333 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.975334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.975404 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.975407 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.975452 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.975487 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":6}
I20260809 18:54:14.975555 15500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.973227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:60317 (local address 127.23.218.194:38725)
0809 18:54:14.973277 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.973279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.973286 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:54:14.973395 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.973395 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.973404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.973449 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:14.973572 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.974013 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.974908 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.975270 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.975329 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.975338 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.975463 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.975496 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:54:14.975765 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.973195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39791 (local address 127.23.218.195:42145)
0809 18:54:14.973345 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:14.973354 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:14.973411 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:14.973561 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:14.973562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:14.973563 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:14.973630 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.973631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.974236 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.974236 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.974999 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:14.975001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.975535 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.975535 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:14.975602 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.975606 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:54:14.975659 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 18:54:14.975695 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260809 18:54:14.975766 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.973278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:42145 (local address 127.23.218.193:39791)
0809 18:54:14.973317 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.973319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.973413 (+ 94us) server_negotiation.cc:407] Connection header received
0809 18:54:14.973462 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.973463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.973486 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.973545 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:14.973686 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.974158 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.975072 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.975463 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.975525 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.975587 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.975666 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.975707 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:54:14.975855 15198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c04f52d45344a1891537e2e8a990047" candidate_uuid: "57e0b19e67d448728b16a8146c33ada6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723df2676ee847a9b8ca07b399413359" is_pre_election: true
I20260809 18:54:14.975980 15198 raft_consensus.cc:2454] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e0b19e67d448728b16a8146c33ada6 in term 0.
I20260809 18:54:14.976065 14971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c04f52d45344a1891537e2e8a990047" candidate_uuid: "57e0b19e67d448728b16a8146c33ada6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b4a8593d9740aa9b3100d2db8b979c" is_pre_election: true
I20260809 18:54:14.976202 15310 leader_election.cc:302] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e0b19e67d448728b16a8146c33ada6, 723df2676ee847a9b8ca07b399413359; no voters:
I20260809 18:54:14.976218 14971 raft_consensus.cc:2454] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57e0b19e67d448728b16a8146c33ada6 in term 0.
I20260809 18:54:14.976311 15512 raft_consensus.cc:2781] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:54:14.976363 15512 raft_consensus.cc:482] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:54:14.976379 15512 raft_consensus.cc:3037] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:14.976962 15512 raft_consensus.cc:504] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.977066 15512 leader_election.cc:288] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [CANDIDATE]: Term 1 election: Requested vote from peers 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725), 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193:39791)
I20260809 18:54:14.977209 15198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c04f52d45344a1891537e2e8a990047" candidate_uuid: "57e0b19e67d448728b16a8146c33ada6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723df2676ee847a9b8ca07b399413359"
I20260809 18:54:14.977210 14971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c04f52d45344a1891537e2e8a990047" candidate_uuid: "57e0b19e67d448728b16a8146c33ada6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20b4a8593d9740aa9b3100d2db8b979c"
I20260809 18:54:14.977288 15198 raft_consensus.cc:3037] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:14.977288 14971 raft_consensus.cc:3037] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:14.978072 15198 raft_consensus.cc:2454] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e0b19e67d448728b16a8146c33ada6 in term 1.
I20260809 18:54:14.978072 14971 raft_consensus.cc:2454] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57e0b19e67d448728b16a8146c33ada6 in term 1.
I20260809 18:54:14.978221 15310 leader_election.cc:302] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57e0b19e67d448728b16a8146c33ada6, 723df2676ee847a9b8ca07b399413359; no voters:
I20260809 18:54:14.978351 15512 raft_consensus.cc:2781] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:14.978467 15512 raft_consensus.cc:686] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 1 LEADER]: Becoming Leader. State: Replica: 57e0b19e67d448728b16a8146c33ada6, State: Running, Role: LEADER
I20260809 18:54:14.978569 15512 consensus_queue.cc:237] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:14.979252 14731 catalog_manager.cc:5495] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195). New cstate: current_term: 1 leader_uuid: "57e0b19e67d448728b16a8146c33ada6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } health_report { overall_health: UNKNOWN } } }
I20260809 18:54:14.985045 15501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:14.983297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43214 (local address 127.23.218.195:44939)
0809 18:54:14.983345 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:14.983346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:14.983352 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:54:14.983375 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:14.983376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:14.983388 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:14.983437 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:54:14.983523 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.983926 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.984540 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:14.984701 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:14.984746 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:14.984835 (+ 89us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:54:14.984917 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:54:14.984966 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:14.984977 (+ 11us) server_negotiation.cc:299] Negotiation successful
0809 18:54:14.984989 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:54:14.986476 14971 raft_consensus.cc:1261] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Refusing update from remote peer 57e0b19e67d448728b16a8146c33ada6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:14.986506 15198 raft_consensus.cc:1261] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [term 1 FOLLOWER]: Refusing update from remote peer 57e0b19e67d448728b16a8146c33ada6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:14.986758 15515 consensus_queue.cc:1035] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:14.986873 15512 consensus_queue.cc:1035] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:14.988876 15525 mvcc.cc:205] Tried to move back new op lower bound from 7316691578822516736 to 7316691578792869888. Current Snapshot: MvccSnapshot[applied={T|T < 7316691578822516736}]
I20260809 18:54:14.989423 15523 mvcc.cc:205] Tried to move back new op lower bound from 7316691578822516736 to 7316691578792869888. Current Snapshot: MvccSnapshot[applied={T|T < 7316691578822516736}]
I20260809 18:54:14.991031 14729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:14.991117 14729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:14.991750 14729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Sending DeleteTablet for 3 replicas of tablet 1c04f52d45344a1891537e2e8a990047
I20260809 18:54:14.992020 15392 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c04f52d45344a1891537e2e8a990047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:14 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:43202
I20260809 18:54:14.992049 14904 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c04f52d45344a1891537e2e8a990047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:14 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:35590
I20260809 18:54:14.992126 15061 tablet_replica.cc:330] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: stopping tablet replica
I20260809 18:54:14.992167 15061 raft_consensus.cc:2229] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:14.992138 15164 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c04f52d45344a1891537e2e8a990047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:14 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:34760
I20260809 18:54:14.992203 15061 raft_consensus.cc:2258] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:14.992225 15537 tablet_replica.cc:330] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: stopping tablet replica
I20260809 18:54:14.992393 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 14795
I20260809 18:54:14.992514 15061 ts_tablet_manager.cc:1905] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:14.992900 15537 raft_consensus.cc:2229] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:54:14.993085 15537 raft_consensus.cc:2258] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:14.996702 15537 ts_tablet_manager.cc:1905] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:14.998564 15537 ts_tablet_manager.cc:1918] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:54:14.998656 15537 log.cc:1196] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/1c04f52d45344a1891537e2e8a990047
I20260809 18:54:14.999015 15537 ts_tablet_manager.cc:1939] T 1c04f52d45344a1891537e2e8a990047 P 57e0b19e67d448728b16a8146c33ada6: Deleting consensus metadata
I20260809 18:54:15.005707 15538 tablet_replica.cc:330] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: stopping tablet replica
I20260809 18:54:15.005832 15538 raft_consensus.cc:2229] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:15.005836 14698 catalog_manager.cc:4841] TS 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939): tablet 1c04f52d45344a1891537e2e8a990047 (table test-table [id=cc06df3055274231879b31eff59a06c1]) successfully deleted
I20260809 18:54:15.005937 15538 raft_consensus.cc:2258] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:15.007068 15538 ts_tablet_manager.cc:1905] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:15.008246 15538 ts_tablet_manager.cc:1918] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:54:15.008311 15538 log.cc:1196] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/1c04f52d45344a1891537e2e8a990047
I20260809 18:54:15.008559 15538 ts_tablet_manager.cc:1939] T 1c04f52d45344a1891537e2e8a990047 P 723df2676ee847a9b8ca07b399413359: Deleting consensus metadata
I20260809 18:54:15.008908 14697 catalog_manager.cc:4841] TS 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725): tablet 1c04f52d45344a1891537e2e8a990047 (table test-table [id=cc06df3055274231879b31eff59a06c1]) successfully deleted
W20260809 18:54:15.012058 14698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:39791 (error 108)
I20260809 18:54:15.012456 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:15.012346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39791 (local address 127.0.0.1:35594)
0809 18:54:15.012389 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.012423 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:39791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 18:54:15.012471 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 15053
W20260809 18:54:15.024037 14698 catalog_manager.cc:4584] TS 20b4a8593d9740aa9b3100d2db8b979c (127.23.218.193:39791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c04f52d45344a1891537e2e8a990047: Network error: Client connection negotiation failed: client connection to 127.23.218.193:39791: connect: Connection refused (error 111)
I20260809 18:54:15.027326 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 15286
I20260809 18:54:15.036773 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 14674
I20260809 18:54:15.044916 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38173
--webserver_interface=127.23.218.254
--webserver_port=35487
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:15.122713 15541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:15.122885 15541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:15.122905 15541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:15.123987 15541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:15.124027 15541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:15.124040 15541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:15.124063 15541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:15.124085 15541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:15.125499 15541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=35487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:15.125707 15541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:15.125918 15541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:15.130334 15547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:15.130317 15550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:15.130362 15548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:15.130642 15541 server_base.cc:1034] running on GCE node
I20260809 18:54:15.130837 15541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:15.131049 15541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:15.132179 15541 hybrid_clock.cc:648] HybridClock initialized: now 1786301655132167 us; error 26 us; skew 500 ppm
Aug 09 18:54:15 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301655, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:15.134392 15541 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:54:15.134759 15541 webserver.cc:460] Webserver started at http://127.23.218.254:35487/ using document root <none> and password file <none>
I20260809 18:54:15.134963 15541 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:15.135008 15541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:15.136169 15541 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:15.136708 15558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.136855 15541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:15.136924 15541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "a0b3e4d6a65f49299690e11d502377e8"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "1e06275f14aac70f319854cbfd53b907"
server_key_iv: "d1d10ee465fbd3ebfb7c7e4acd6ce8ec"
server_key_version: "encryptionkey@0"
I20260809 18:54:15.137215 15541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:15.146130 15541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:15.147665 15541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:15.147782 15541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:15.151412 15541 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:38173
I20260809 18:54:15.151440 15642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:38173 every 8 connection(s)
I20260809 18:54:15.151742 15541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:54:15.153067 15644 sys_catalog.cc:263] Verifying existing consensus state
I20260809 18:54:15.153650 15644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Bootstrap starting.
I20260809 18:54:15.159312 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 15541
I20260809 18:54:15.159626 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:39791
--local_ip_for_outbound_sockets=127.23.218.193
--tserver_master_addrs=127.23.218.254:38173
--webserver_port=44619
--webserver_interface=127.23.218.193
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:54:15.162513 15644 log.cc:824] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:15.164485 15644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=7 ignored=0} mutations{seen=10 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:54:15.164731 15644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8: Bootstrap complete.
I20260809 18:54:15.166761 15644 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:15.166965 15644 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0b3e4d6a65f49299690e11d502377e8, State: Initialized, Role: FOLLOWER
I20260809 18:54:15.167089 15644 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 2.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:15.167145 15644 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:54:15.167183 15644 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:54:15.167227 15644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 2 FOLLOWER]: Advancing to term 3
I20260809 18:54:15.168095 15644 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:15.168179 15644 leader_election.cc:302] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0b3e4d6a65f49299690e11d502377e8; no voters:
I20260809 18:54:15.168315 15644 leader_election.cc:288] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 18:54:15.168361 15650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 18:54:15.168474 15650 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [term 3 LEADER]: Becoming Leader. State: Replica: a0b3e4d6a65f49299690e11d502377e8, State: Running, Role: LEADER
I20260809 18:54:15.168557 15644 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:15.168563 15650 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 2.15, Last appended by leader: 15, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } }
I20260809 18:54:15.168826 15650 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0b3e4d6a65f49299690e11d502377e8. Latest consensus state: current_term: 3 leader_uuid: "a0b3e4d6a65f49299690e11d502377e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } } }
I20260809 18:54:15.168885 15650 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:15.169040 15651 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a0b3e4d6a65f49299690e11d502377e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0b3e4d6a65f49299690e11d502377e8" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38173 } } }
I20260809 18:54:15.169188 15651 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0b3e4d6a65f49299690e11d502377e8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:15.169412 15655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:54:15.170128 15655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:54:15.170220 15655 catalog_manager.cc:1259] Loaded cluster ID: c73d8d5399a14430850eea3b74832051
I20260809 18:54:15.170241 15655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 18:54:15.170573 15655 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f927c7439ad google::LogMessage::Fail() at ??:0
@ 0x7f927c74688a google::LogMessage::SendToLog() at ??:0
@ 0x7f927c7434de google::LogMessage::Flush() at ??:0
@ 0x7f927c744a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f92846640f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f928466d8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f927e6110d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f927e606954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f927f46a6db start_thread at ??:0
@ 0x7f927b24161f clone at ??:0
W20260809 18:54:15.268610 15648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:15.268800 15648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:15.268831 15648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:15.269919 15648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:15.269956 15648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:15.269991 15648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:54:15.271427 15648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:39791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=44619
--tserver_master_addrs=127.23.218.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:15.271642 15648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:15.271842 15648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:15.276463 15695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:15.276499 15699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:15.276464 15694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:15.276768 15648 server_base.cc:1034] running on GCE node
I20260809 18:54:15.277017 15648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:15.277278 15648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:15.278391 15648 hybrid_clock.cc:648] HybridClock initialized: now 1786301655278377 us; error 30 us; skew 500 ppm
Aug 09 18:54:15 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301655, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:15.280820 15648 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:54:15.281293 15648 webserver.cc:460] Webserver started at http://127.23.218.193:44619/ using document root <none> and password file <none>
I20260809 18:54:15.281507 15648 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:15.281553 15648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:15.282866 15648 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:15.283613 15709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.283771 15648 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:15.283856 15648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "20b4a8593d9740aa9b3100d2db8b979c"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "186f4908c61ed6e95f7a584fd3f261e9"
server_key_iv: "85dbedbc03dd7cc12fd3249d5c3cbfaf"
server_key_version: "encryptionkey@0"
I20260809 18:54:15.284165 15648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:15.304168 15648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:15.306126 15648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:15.306253 15648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:15.306483 15648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:15.306885 15722 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 18:54:15.308699 15648 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 18:54:15.308748 15648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 18:54:15.308779 15648 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 18:54:15.309401 15648 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 18:54:15.309433 15648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 18:54:15.309525 15722 tablet_bootstrap.cc:492] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Bootstrap starting.
I20260809 18:54:15.312379 15722 log.cc:824] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:15.315164 15722 tablet_bootstrap.cc:492] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:54:15.315524 15722 tablet_bootstrap.cc:492] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Bootstrap complete.
I20260809 18:54:15.315686 15722 ts_tablet_manager.cc:1397] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 18:54:15.316098 15648 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:39791
I20260809 18:54:15.316406 15648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:54:15.318018 15722 raft_consensus.cc:348] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:15.318320 15722 raft_consensus.cc:729] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b4a8593d9740aa9b3100d2db8b979c, State: Initialized, Role: FOLLOWER
I20260809 18:54:15.318465 15722 consensus_queue.cc:260] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:15.318722 15722 ts_tablet_manager.cc:1428] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 18:54:15.318948 15912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:39791 every 8 connection(s)
I20260809 18:54:15.325402 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 15648
I20260809 18:54:15.325644 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:38725
--local_ip_for_outbound_sockets=127.23.218.194
--tserver_master_addrs=127.23.218.254:38173
--webserver_port=39091
--webserver_interface=127.23.218.194
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:15.369601 15916 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:54:15.317031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:40927)
0809 18:54:15.317178 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.317209 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:15.317995 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:15.369417 (+ 51422us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
W20260809 18:54:15.369845 15913 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:38173 (0 consecutive failures): Network error: Failed to ping master at 127.23.218.254:38173: Client connection negotiation failed: client connection to 127.23.218.254:38173: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 18:54:15.370172 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:15.370055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:53341)
0809 18:54:15.370111 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.370151 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 18:54:15.370456 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:15.370394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:35767)
0809 18:54:15.370420 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.370441 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260809 18:54:15.370556 15913 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 18:54:15.407262 15919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:15.407438 15919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:15.407469 15919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:15.408558 15919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:15.408603 15919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:15.408638 15919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:54:15.410032 15919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:38725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=39091
--tserver_master_addrs=127.23.218.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:15.410240 15919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:15.410444 15919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:15.414855 15933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:15.414860 15926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:15.414892 15928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:15.415086 15919 server_base.cc:1034] running on GCE node
I20260809 18:54:15.415328 15919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:15.415535 15919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:15.416668 15919 hybrid_clock.cc:648] HybridClock initialized: now 1786301655416653 us; error 30 us; skew 500 ppm
Aug 09 18:54:15 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301655, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:15.418937 15919 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:54:15.419273 15919 webserver.cc:460] Webserver started at http://127.23.218.194:39091/ using document root <none> and password file <none>
I20260809 18:54:15.419426 15919 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:15.419462 15919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:15.420594 15919 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:15.421164 15944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.421319 15919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.421388 15919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "723df2676ee847a9b8ca07b399413359"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "ed8eba22cbec483cd1dd0a1ba5a7a666"
server_key_iv: "fe06cb09640780345a0f7bdf8e8dfcc0"
server_key_version: "encryptionkey@0"
I20260809 18:54:15.421671 15919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:15.451643 15919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:15.453178 15919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:15.453289 15919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:15.453477 15919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:15.453730 15919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:15.453763 15919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.453780 15919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:15.453804 15919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.459177 15919 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:38725
I20260809 18:54:15.459246 16127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:38725 every 8 connection(s)
I20260809 18:54:15.459499 15919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:54:15.460367 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:15.460019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:39433)
0809 18:54:15.460148 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.460223 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
W20260809 18:54:15.460690 16130 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:38173 (0 consecutive failures): Network error: Failed to ping master at 127.23.218.254:38173: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
I20260809 18:54:15.460933 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:15.460842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:52213)
0809 18:54:15.460882 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.460911 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 18:54:15.460923 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 15919
I20260809 18:54:15.461184 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:15.461107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:48179)
0809 18:54:15.461141 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.461165 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 18:54:15.461190 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:44939
--local_ip_for_outbound_sockets=127.23.218.195
--tserver_master_addrs=127.23.218.254:38173
--webserver_port=40211
--webserver_interface=127.23.218.195
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:15.461349 16130 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 18:54:15.539873 16136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:15.540041 16136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:15.540062 16136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:15.541103 16136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:15.541134 16136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:15.541158 16136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:54:15.542523 16136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:44939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=40211
--tserver_master_addrs=127.23.218.254:38173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:15.542695 16136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:15.542847 16136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:15.547402 16143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:15.547399 16142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:15.547412 16145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:15.547626 16136 server_base.cc:1034] running on GCE node
I20260809 18:54:15.547811 16136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:15.548017 16136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:15.549149 16136 hybrid_clock.cc:648] HybridClock initialized: now 1786301655549134 us; error 28 us; skew 500 ppm
Aug 09 18:54:15 dist-test-slave-6zbq krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301655, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:15.551388 16136 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:54:15.551769 16136 webserver.cc:460] Webserver started at http://127.23.218.195:40211/ using document root <none> and password file <none>
I20260809 18:54:15.551946 16136 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:15.551991 16136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:15.553148 16136 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:15.553637 16159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.553777 16136 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.553857 16136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "57e0b19e67d448728b16a8146c33ada6"
format_stamp: "Formatted at 2026-08-09 18:54:13 on dist-test-slave-6zbq"
server_key: "30538104bd5fdcf0c9337f56c64b9efd"
server_key_iv: "ee945a8a6611f0be433df1db59192aa2"
server_key_version: "encryptionkey@0"
I20260809 18:54:15.554132 16136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:15.567552 16136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:15.569011 16136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:15.569135 16136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:15.569316 16136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:15.569562 16136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:15.569594 16136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.569619 16136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:15.569635 16136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:15.574891 16136 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:44939
I20260809 18:54:15.574924 16345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:44939 every 8 connection(s)
I20260809 18:54:15.575223 16136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:54:15.576025 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 16136
I20260809 18:54:15.576166 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:15.575768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:39789)
0809 18:54:15.575906 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.575994 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
W20260809 18:54:15.576485 16347 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:38173 (0 consecutive failures): Network error: Failed to ping master at 127.23.218.254:38173: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
I20260809 18:54:15.576723 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:15.576644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:52243)
0809 18:54:15.576683 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.576706 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 18:54:15.576951 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:15.576878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:45657)
0809 18:54:15.576910 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:15.576930 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 18:54:15.577031 16347 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 18:54:16.371016 16357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:16.370821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:52217)
0809 18:54:16.370934 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:16.370965 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260809 18:54:16.461796 16358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:16.461604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:36145)
0809 18:54:16.461715 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:16.461750 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260809 18:54:16.540002 16359 raft_consensus.cc:482] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:16.540117 16359 raft_consensus.cc:504] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:16.540462 16359 leader_election.cc:288] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939), 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725)
Aug 09 18:54:16 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301655, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.194@KRBTEST.COM
Aug 09 18:54:16 dist-test-slave-6zbq krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301655, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.195@KRBTEST.COM
I20260809 18:54:16.550267 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:16.540735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:43181 (local address 127.23.218.195:44939)
0809 18:54:16.540874 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:16.540883 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:16.540903 (+ 20us) server_negotiation.cc:407] Connection header received
0809 18:54:16.541311 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:16.541313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:16.541374 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:16.541497 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:16.542181 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:16.543138 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:16.544113 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:16.544515 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:16.544615 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:16.547233 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:16.547237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:16.547238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:16.547249 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:16.548717 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:16.549058 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:16.549061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:16.549061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:16.549161 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:16.549343 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:16.549343 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:16.549344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:16.549647 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:16.549725 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:16.550036 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 18:54:16.550173 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
I20260809 18:54:16.550279 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:16.540681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:35649 (local address 127.23.218.194:38725)
0809 18:54:16.540848 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:16.540859 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:16.540889 (+ 30us) server_negotiation.cc:407] Connection header received
0809 18:54:16.541329 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:16.541331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:16.541367 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:16.541484 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:16.542254 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:16.543121 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:16.544089 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:16.544503 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:16.544593 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:16.546832 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:16.546836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:16.546836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:16.546847 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:16.548537 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:16.548977 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:16.548980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:16.548981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:16.549097 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:16.549375 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:16.549376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:16.549377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:16.549642 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:16.549726 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:16.550036 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 18:54:16.550172 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":60,"threads_started":1}
I20260809 18:54:16.550268 16357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:16.540589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:38725 (local address 127.23.218.193:35649)
0809 18:54:16.540689 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:16.540701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:16.540766 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:16.541543 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:16.541546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:16.542075 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:16.542146 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:16.542149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:16.543219 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:16.543220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:16.544009 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:16.544012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:16.544593 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:16.544594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:16.544714 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:16.545378 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:16.545383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:16.546733 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:16.548610 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:16.548618 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:16.548620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:16.548902 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:16.549159 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:16.549160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:16.549160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:16.549304 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:16.549724 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:16.549725 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:16.549865 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 18:54:16.549974 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 18:54:16.550140 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":23}
I20260809 18:54:16.550747 16360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:16.540590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:44939 (local address 127.23.218.193:43181)
0809 18:54:16.540698 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:16.540708 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:16.540764 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:16.541544 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:16.541546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:16.541843 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:16.542030 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:16.542036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:16.543253 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:16.543254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:16.544009 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:16.544011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:16.544594 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:16.544594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:16.544713 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:16.545326 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:16.545335 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:16.547149 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:16.548792 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:16.548793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:16.548793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:16.548980 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:16.549225 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:16.549233 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:16.549234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:16.549285 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:16.549708 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:16.549716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:16.549865 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 18:54:16.549974 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 18:54:16.550204 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":158,"thread_start_us":46,"threads_started":1}
W20260809 18:54:16.551136 15710 leader_election.cc:341] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725): Not found: Tablet not found: 1c04f52d45344a1891537e2e8a990047
W20260809 18:54:16.551409 15711 leader_election.cc:341] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939): Not found: Tablet not found: 1c04f52d45344a1891537e2e8a990047
I20260809 18:54:16.551461 15711 leader_election.cc:302] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20b4a8593d9740aa9b3100d2db8b979c; no voters: 57e0b19e67d448728b16a8146c33ada6, 723df2676ee847a9b8ca07b399413359
I20260809 18:54:16.551554 16359 raft_consensus.cc:2726] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:54:16.577445 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:16.577269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:36993)
0809 18:54:16.577379 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:16.577415 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260809 18:54:17.371582 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:17.371379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:35837)
0809 18:54:17.371495 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:17.371532 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260809 18:54:17.462401 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:17.462188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:39859)
0809 18:54:17.462300 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:17.462334 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260809 18:54:17.577984 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:17.577794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:43033)
0809 18:54:17.577916 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:17.577949 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260809 18:54:18.372120 16373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:18.371937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:39261)
0809 18:54:18.372040 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:18.372077 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260809 18:54:18.462927 16374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:18.462751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:43439)
0809 18:54:18.462858 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:18.462889 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260809 18:54:18.578598 16375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:18.578392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:56613)
0809 18:54:18.578516 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:18.578549 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260809 18:54:18.720976 16378 raft_consensus.cc:482] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:18.721081 16378 raft_consensus.cc:504] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:18.721278 16378 leader_election.cc:288] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939), 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725)
W20260809 18:54:18.721649 15711 leader_election.cc:341] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939): Not found: Tablet not found: 1c04f52d45344a1891537e2e8a990047
W20260809 18:54:18.721769 15710 leader_election.cc:341] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725): Not found: Tablet not found: 1c04f52d45344a1891537e2e8a990047
I20260809 18:54:18.721844 15710 leader_election.cc:302] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20b4a8593d9740aa9b3100d2db8b979c; no voters: 57e0b19e67d448728b16a8146c33ada6, 723df2676ee847a9b8ca07b399413359
I20260809 18:54:18.721928 16378 raft_consensus.cc:2726] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:54:19.372679 16381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:19.372490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:48611)
0809 18:54:19.372603 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:19.372636 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260809 18:54:19.463443 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:19.463265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:45891)
0809 18:54:19.463375 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:19.463409 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260809 18:54:19.579177 16383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:19.578982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:52361)
0809 18:54:19.579099 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:19.579140 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260809 18:54:20.373308 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:20.373104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:53735)
0809 18:54:20.373222 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:20.373264 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260809 18:54:20.464068 16385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:20.463836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:53091)
0809 18:54:20.463988 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:20.464030 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":58,"threads_started":1}
I20260809 18:54:20.579726 16386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:20.579521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:60893)
0809 18:54:20.579661 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:20.579690 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260809 18:54:21.373863 16387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:21.373670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:53755)
0809 18:54:21.373787 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:21.373820 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260809 18:54:21.464686 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:21.464497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:44719)
0809 18:54:21.464616 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:21.464651 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":55,"threads_started":1}
I20260809 18:54:21.580291 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:21.580087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:44265)
0809 18:54:21.580212 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:21.580252 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260809 18:54:21.736788 16390 raft_consensus.cc:482] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:21.736881 16390 raft_consensus.cc:504] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:21.737046 16390 leader_election.cc:288] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939), 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725)
W20260809 18:54:21.737397 15710 leader_election.cc:341] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725): Not found: Tablet not found: 1c04f52d45344a1891537e2e8a990047
W20260809 18:54:21.737483 15711 leader_election.cc:341] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939): Not found: Tablet not found: 1c04f52d45344a1891537e2e8a990047
I20260809 18:54:21.737524 15711 leader_election.cc:302] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20b4a8593d9740aa9b3100d2db8b979c; no voters: 57e0b19e67d448728b16a8146c33ada6, 723df2676ee847a9b8ca07b399413359
I20260809 18:54:21.737586 16390 raft_consensus.cc:2726] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:54:22.374516 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:22.374280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:50977)
0809 18:54:22.374437 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:22.374471 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260809 18:54:22.465240 16392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:22.465061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:48945)
0809 18:54:22.465175 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:22.465204 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260809 18:54:22.580838 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:22.580656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:42811)
0809 18:54:22.580771 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:22.580803 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260809 18:54:23.375083 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:23.374889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:49247)
0809 18:54:23.375005 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:23.375037 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260809 18:54:23.465777 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:23.465578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:36207)
0809 18:54:23.465704 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:23.465740 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260809 18:54:23.581416 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:23.581217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:59457)
0809 18:54:23.581342 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:23.581376 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260809 18:54:24.375689 16402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:24.375494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:34981)
0809 18:54:24.375611 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:24.375642 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260809 18:54:24.466306 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:24.466115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:51955)
0809 18:54:24.466236 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:24.466268 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260809 18:54:24.582010 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:24.581812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:32917)
0809 18:54:24.581935 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:24.581968 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
I20260809 18:54:25.376308 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:25.376095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:40811)
0809 18:54:25.376228 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:25.376262 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260809 18:54:25.466835 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:25.466657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:39347)
0809 18:54:25.466761 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:25.466794 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260809 18:54:25.582569 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:25.582402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:43237)
0809 18:54:25.582501 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:25.582533 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260809 18:54:26.265808 16410 raft_consensus.cc:482] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:26.265908 16410 raft_consensus.cc:504] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57e0b19e67d448728b16a8146c33ada6" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 44939 } } peers { permanent_uuid: "723df2676ee847a9b8ca07b399413359" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 38725 } } peers { permanent_uuid: "20b4a8593d9740aa9b3100d2db8b979c" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39791 } }
I20260809 18:54:26.266090 16410 leader_election.cc:288] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939), 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725)
W20260809 18:54:26.266465 15711 leader_election.cc:341] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57e0b19e67d448728b16a8146c33ada6 (127.23.218.195:44939): Not found: Tablet not found: 1c04f52d45344a1891537e2e8a990047
W20260809 18:54:26.266543 15710 leader_election.cc:341] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 723df2676ee847a9b8ca07b399413359 (127.23.218.194:38725): Not found: Tablet not found: 1c04f52d45344a1891537e2e8a990047
I20260809 18:54:26.266615 15710 leader_election.cc:302] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20b4a8593d9740aa9b3100d2db8b979c; no voters: 57e0b19e67d448728b16a8146c33ada6, 723df2676ee847a9b8ca07b399413359
I20260809 18:54:26.266703 16410 raft_consensus.cc:2726] T 1c04f52d45344a1891537e2e8a990047 P 20b4a8593d9740aa9b3100d2db8b979c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:54:26.376894 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:26.376680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:51197)
0809 18:54:26.376818 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:26.376850 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260809 18:54:26.467451 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:26.467224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:32871)
0809 18:54:26.467361 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:26.467395 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260809 18:54:26.583173 16416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:26.582962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:44035)
0809 18:54:26.583088 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:26.583117 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260809 18:54:27.377540 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:27.377319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:54631)
0809 18:54:27.377454 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:27.377492 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260809 18:54:27.467995 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:27.467822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:55843)
0809 18:54:27.467927 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:27.467959 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":58,"threads_started":1}
I20260809 18:54:27.583779 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:27.583588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:35487)
0809 18:54:27.583705 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:27.583740 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
I20260809 18:54:28.378162 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:28.377951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:36029)
0809 18:54:28.378062 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:28.378118 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260809 18:54:28.468533 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:28.468349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:44621)
0809 18:54:28.468461 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:28.468494 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260809 18:54:28.584333 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:28.584131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:55447)
0809 18:54:28.584262 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:28.584294 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260809 18:54:29.378851 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:29.378655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:39725)
0809 18:54:29.378772 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:29.378807 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260809 18:54:29.469122 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:29.468913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:34095)
0809 18:54:29.469033 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:29.469067 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260809 18:54:29.584856 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:29.584677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:49843)
0809 18:54:29.584786 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:29.584822 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260809 18:54:30.379498 16434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:30.379313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.193:59549)
0809 18:54:30.379426 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:30.379455 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260809 18:54:30.469833 16436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:30.469613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.194:53633)
0809 18:54:30.469758 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:30.469793 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260809 18:54:30.578823 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 15648
I20260809 18:54:30.587201 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:30.586019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38173 (local address 127.23.218.195:53083)
0809 18:54:30.587135 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:30.587165 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1101,"thread_start_us":1058,"threads_started":1}
I20260809 18:54:30.591452 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 15919
I20260809 18:54:30.600445 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 16136
2026-08-09T18:54:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18474 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:54:30 dist-test-slave-6zbq krb5kdc[16443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:54:30 dist-test-slave-6zbq krb5kdc[16443](info): set up 2 sockets
Aug 09 18:54:30 dist-test-slave-6zbq krb5kdc[16443](info): commencing operation
krb5kdc: starting...
W20260809 18:54:31.731817 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:54:31 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:54:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:54:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:54:31.835163 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38087
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:31.914494 16466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:31.914671 16466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:31.914700 16466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:31.915798 16466 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 18:54:31.915835 16466 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 18:54:31.915853 16466 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 18:54:31.915867 16466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:31.915884 16466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:31.915907 16466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:31.915930 16466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:31.915947 16466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:31.917384 16466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:31.917594 16466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:31.917779 16466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:31.922327 16473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:31.922399 16472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:31.922319 16476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:31.922391 16466 server_base.cc:1034] running on GCE node
I20260809 18:54:31.922747 16466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:31.922994 16466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:31.924131 16466 hybrid_clock.cc:648] HybridClock initialized: now 1786301671924116 us; error 27 us; skew 500 ppm
Aug 09 18:54:31 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301671, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:31.926433 16466 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:54:31.926846 16466 webserver.cc:460] Webserver started at http://127.23.218.254:36659/ using document root <none> and password file <none>
I20260809 18:54:31.927055 16466 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:31.927103 16466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:31.927227 16466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:31.928179 16466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "9a344976f7a4471fba16c5e5e1d3ac4b"
format_stamp: "Formatted at 2026-08-09 18:54:31 on dist-test-slave-6zbq"
server_key: "c1979473a99622adb6608956619e9ba6"
server_key_iv: "8946d1807db67fbb748c761087e5a5c8"
server_key_version: "encryptionkey@0"
I20260809 18:54:31.928494 16466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "9a344976f7a4471fba16c5e5e1d3ac4b"
format_stamp: "Formatted at 2026-08-09 18:54:31 on dist-test-slave-6zbq"
server_key: "c1979473a99622adb6608956619e9ba6"
server_key_iv: "8946d1807db67fbb748c761087e5a5c8"
server_key_version: "encryptionkey@0"
I20260809 18:54:31.929706 16466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:54:31.930325 16486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:31.930470 16466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:31.930554 16466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "9a344976f7a4471fba16c5e5e1d3ac4b"
format_stamp: "Formatted at 2026-08-09 18:54:31 on dist-test-slave-6zbq"
server_key: "c1979473a99622adb6608956619e9ba6"
server_key_iv: "8946d1807db67fbb748c761087e5a5c8"
server_key_version: "encryptionkey@0"
I20260809 18:54:31.930631 16466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:31.945576 16466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:31.947562 16466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:31.947696 16466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:31.951365 16466 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:38087
I20260809 18:54:31.951404 16572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:38087 every 8 connection(s)
I20260809 18:54:31.951714 16466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:54:31.952486 16573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:31.954577 16573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Bootstrap starting.
I20260809 18:54:31.955119 16573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:31.955390 16573 log.cc:824] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:31.955991 16573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: No bootstrap required, opened a new log
I20260809 18:54:31.957505 16573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:54:31.957630 16573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:31.957661 16573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a344976f7a4471fba16c5e5e1d3ac4b, State: Initialized, Role: FOLLOWER
I20260809 18:54:31.957798 16573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:54:31.957858 16573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:54:31.957880 16573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:54:31.957926 16573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:31.958439 16573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:54:31.958530 16573 leader_election.cc:302] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a344976f7a4471fba16c5e5e1d3ac4b; no voters:
I20260809 18:54:31.958676 16573 leader_election.cc:288] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:54:31.958706 16578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:31.958827 16578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 1 LEADER]: Becoming Leader. State: Replica: 9a344976f7a4471fba16c5e5e1d3ac4b, State: Running, Role: LEADER
I20260809 18:54:31.958916 16573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:31.959075 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 16466
I20260809 18:54:31.959179 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:54:31.958958 16578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:54:31.959383 24427 external_mini_cluster.cc:1428] Setting key ebbdbe5983bc08879c4aa37c4bb4b18c
I20260809 18:54:31.959518 16579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } } }
I20260809 18:54:31.959755 16579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:31.959688 16580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a344976f7a4471fba16c5e5e1d3ac4b. Latest consensus state: current_term: 1 leader_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } } }
I20260809 18:54:31.960021 16580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:31.960350 16598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:54:31.960947 16598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:54:31.962399 16598 catalog_manager.cc:1347] Generated new cluster ID: 4bcc72dfe4674485a6b367a9b779b96f
I20260809 18:54:31.962450 16598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:54:31 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:31.969790 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:31.960047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.23.218.254:38087)
0809 18:54:31.960238 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:31.960247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:31.960280 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:54:31.960391 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:31.960398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:31.960451 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:31.960590 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:31.961199 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:31.962155 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:31.963218 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:31.963839 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:31.963953 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:31.966498 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:31.966501 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:31.966502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:31.966512 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:31.968329 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:31.968702 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:31.968704 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:31.968705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:31.968803 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:31.968985 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:31.968985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:31.968986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:31.969239 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:31.969356 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:31.969465 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 18:54:31.969586 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260809 18:54:31.973338 16598 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:54:31.973789 16598 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:54:31.979321 16598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Generated new TSK 0
I20260809 18:54:31.979458 16598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:54:32.004387 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38087
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:32.082300 16624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:32.082489 16624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:32.082510 16624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:32.083552 16624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:32.083582 16624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:32.083612 16624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:54:32.084942 16624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:32.085111 16624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:32.085268 16624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:32.089582 16630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:32.089591 16633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:32.089591 16631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:32.089720 16624 server_base.cc:1034] running on GCE node
I20260809 18:54:32.089986 16624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:32.090185 16624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:32.091311 16624 hybrid_clock.cc:648] HybridClock initialized: now 1786301672091288 us; error 40 us; skew 500 ppm
Aug 09 18:54:32 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301672, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:32.093502 16624 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:54:32.093878 16624 webserver.cc:460] Webserver started at http://127.23.218.193:40103/ using document root <none> and password file <none>
I20260809 18:54:32.094053 16624 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:32.094089 16624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:32.094178 16624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:32.095082 16624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "7264dd200dcd4911a62df6df6f54f136"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "80b235e873a24e8197f885088cd93693"
server_key_iv: "5227b330ec70860f90235207d78b81b9"
server_key_version: "encryptionkey@0"
I20260809 18:54:32.095352 16624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "7264dd200dcd4911a62df6df6f54f136"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "80b235e873a24e8197f885088cd93693"
server_key_iv: "5227b330ec70860f90235207d78b81b9"
server_key_version: "encryptionkey@0"
I20260809 18:54:32.096423 16624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:32.097002 16646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.097157 16624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.097237 16624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "7264dd200dcd4911a62df6df6f54f136"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "80b235e873a24e8197f885088cd93693"
server_key_iv: "5227b330ec70860f90235207d78b81b9"
server_key_version: "encryptionkey@0"
I20260809 18:54:32.097289 16624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:32.104899 16624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:32.106393 16624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:32.106495 16624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:32.106648 16624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:32.106873 16624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:32.106900 16624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.106925 16624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:32.106940 16624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.112214 16624 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:34423
I20260809 18:54:32.112300 16844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:34423 every 8 connection(s)
I20260809 18:54:32.112524 16624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
Aug 09 18:54:32 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301672, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:32.118244 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 16624
I20260809 18:54:32.118358 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:54:32.118527 24427 external_mini_cluster.cc:1428] Setting key aa981fc2598864abbdd2af22a6f31cb9
I20260809 18:54:32.120390 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.113088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:46985 (local address 127.23.218.254:38087)
0809 18:54:32.113159 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.113161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.113283 (+ 122us) server_negotiation.cc:407] Connection header received
0809 18:54:32.113670 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.113671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.113680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.113717 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:32.114554 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.115035 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.115891 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.116065 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.116106 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.117822 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:32.117824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:32.117824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:32.117831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:32.119204 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:32.119548 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:32.119549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:32.119550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:32.119605 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:32.119816 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:32.119817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:32.119817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:32.119924 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:32.119977 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.120216 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.120303 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 18:54:32.120527 16848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.113064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:46985)
0809 18:54:32.113188 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:32.113219 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:32.113590 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:32.113763 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:32.113766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:32.114264 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:32.114489 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.114494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.115094 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.115094 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.115835 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.115837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.116124 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.116125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.116211 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.116548 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:32.116555 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:32.117730 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:32.119275 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:32.119282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:32.119284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:32.119482 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:32.119657 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:32.119658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:32.119658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:32.119760 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:32.119976 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:32.119982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:32.120102 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 18:54:32.120186 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 18:54:32.120331 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":67,"threads_started":1}
I20260809 18:54:32.120978 16845 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38087
I20260809 18:54:32.121083 16845 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:32.121255 16845 heartbeater.cc:511] Master 127.23.218.254:38087 requested a full tablet report, sending...
I20260809 18:54:32.121676 16521 ts_manager.cc:194] Registered new tserver with Master: 7264dd200dcd4911a62df6df6f54f136 (127.23.218.193:34423)
I20260809 18:54:32.122242 16521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:46985
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:54:32.141469 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38087
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:32.217859 16852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:32.218032 16852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:32.218065 16852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:32.219170 16852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:32.219209 16852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:32.219244 16852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:54:32.220640 16852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:32.220854 16852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:32.221036 16852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:32.225368 16857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:32.225363 16861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:32.225365 16858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:32.225593 16852 server_base.cc:1034] running on GCE node
I20260809 18:54:32.225862 16852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:32.226056 16852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:32.227205 16852 hybrid_clock.cc:648] HybridClock initialized: now 1786301672227188 us; error 32 us; skew 500 ppm
Aug 09 18:54:32 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301672, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:32.229391 16852 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:54:32.229796 16852 webserver.cc:460] Webserver started at http://127.23.218.194:45263/ using document root <none> and password file <none>
I20260809 18:54:32.229984 16852 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:32.230026 16852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:32.230144 16852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:32.231107 16852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "931165f44627438b9bc6ebb7691ce2dd"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "c99ac5d2bb1460c6c22e4ff6e6f89e7e"
server_key_iv: "9b7389e0a235c1d6dd9151dc3408dc6e"
server_key_version: "encryptionkey@0"
I20260809 18:54:32.231408 16852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "931165f44627438b9bc6ebb7691ce2dd"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "c99ac5d2bb1460c6c22e4ff6e6f89e7e"
server_key_iv: "9b7389e0a235c1d6dd9151dc3408dc6e"
server_key_version: "encryptionkey@0"
I20260809 18:54:32.232527 16852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:54:32.233124 16871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.233268 16852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:32.233330 16852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "931165f44627438b9bc6ebb7691ce2dd"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "c99ac5d2bb1460c6c22e4ff6e6f89e7e"
server_key_iv: "9b7389e0a235c1d6dd9151dc3408dc6e"
server_key_version: "encryptionkey@0"
I20260809 18:54:32.233394 16852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:32.240423 16852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:32.242273 16852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:32.242424 16852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:32.242610 16852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:32.242882 16852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:32.242913 16852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.242939 16852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:32.242965 16852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.248103 16852 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:34167
I20260809 18:54:32.248159 17063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:34167 every 8 connection(s)
I20260809 18:54:32.248413 16852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
Aug 09 18:54:32 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301672, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:32.255206 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 16852
I20260809 18:54:32.255316 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:54:32.255491 24427 external_mini_cluster.cc:1428] Setting key e3b0eff8913e4aece80465dcccd2b454
I20260809 18:54:32.256268 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.248971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:53849 (local address 127.23.218.254:38087)
0809 18:54:32.249011 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.249012 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.249208 (+ 196us) server_negotiation.cc:407] Connection header received
0809 18:54:32.249607 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.249608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.249617 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.249651 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:32.250464 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.250953 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.251823 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.252004 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.252045 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.253755 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:32.253757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:32.253757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:32.253764 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:32.254946 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:32.255380 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:32.255382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:32.255382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:32.255435 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:32.255699 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:32.255700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:32.255700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:32.255832 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:32.255892 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.256113 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.256195 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 18:54:32.256438 17066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.248944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:53849)
0809 18:54:32.249084 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:32.249138 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:32.249527 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:32.249702 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:32.249705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:32.250197 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:32.250394 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.250400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.251012 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.251013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.251763 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.251765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.252053 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.252053 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.252151 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.252497 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:32.252505 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:32.253660 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:32.255008 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:32.255012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:32.255013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:32.255307 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:32.255492 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:32.255493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:32.255493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:32.255620 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:32.255887 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:32.255892 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:32.255992 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 18:54:32.256087 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 18:54:32.256245 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260809 18:54:32.256860 17064 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38087
I20260809 18:54:32.256947 17064 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:32.257110 17064 heartbeater.cc:511] Master 127.23.218.254:38087 requested a full tablet report, sending...
I20260809 18:54:32.257428 16521 ts_manager.cc:194] Registered new tserver with Master: 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
I20260809 18:54:32.257778 16521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:53849
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:54:32.276624 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38087
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:32.354553 17073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:32.354730 17073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:32.354750 17073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:32.355820 17073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:32.355852 17073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:32.355875 17073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:54:32.357221 17073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:38087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:32.357376 17073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:32.357535 17073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:32.362174 17080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:32.362170 17085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:32.362170 17082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:32.362329 17073 server_base.cc:1034] running on GCE node
I20260809 18:54:32.362524 17073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:32.362749 17073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:32.363878 17073 hybrid_clock.cc:648] HybridClock initialized: now 1786301672363858 us; error 34 us; skew 500 ppm
Aug 09 18:54:32 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301672, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:32.366103 17073 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:54:32.366544 17073 webserver.cc:460] Webserver started at http://127.23.218.195:43867/ using document root <none> and password file <none>
I20260809 18:54:32.366730 17073 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:32.366776 17073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:32.366897 17073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:54:32.367825 17073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "11af8815e8c800ff2a8bedab56aa38f3"
server_key_iv: "50f927fa48cc69c9fb5bc27a92c205fe"
server_key_version: "encryptionkey@0"
I20260809 18:54:32.368146 17073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "11af8815e8c800ff2a8bedab56aa38f3"
server_key_iv: "50f927fa48cc69c9fb5bc27a92c205fe"
server_key_version: "encryptionkey@0"
I20260809 18:54:32.369330 17073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:54:32.369946 17093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.370091 17073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.370170 17073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "11af8815e8c800ff2a8bedab56aa38f3"
server_key_iv: "50f927fa48cc69c9fb5bc27a92c205fe"
server_key_version: "encryptionkey@0"
I20260809 18:54:32.370240 17073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:32.379570 17073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:32.381079 17073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:32.381196 17073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:32.381383 17073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:32.381631 17073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:32.381664 17073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.381687 17073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:32.381713 17073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:32.386915 17073 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:40459
I20260809 18:54:32.386974 17278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:40459 every 8 connection(s)
I20260809 18:54:32.387203 17073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:54:32.390415 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 17073
I20260809 18:54:32.390498 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:54:32.390679 24427 external_mini_cluster.cc:1428] Setting key 3b85a23fc2e22ad500a1c7817c8012d9
Aug 09 18:54:32 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301672, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:32.395071 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.387782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:49067 (local address 127.23.218.254:38087)
0809 18:54:32.387851 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.387853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.387975 (+ 122us) server_negotiation.cc:407] Connection header received
0809 18:54:32.388358 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.388359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.388369 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.388415 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:32.389181 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.389663 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.390647 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.390911 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.390981 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.392846 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:32.392847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:32.392848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:32.392854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:32.394018 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:32.394305 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:32.394306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:32.394306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:32.394379 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:32.394565 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:32.394565 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:32.394565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:32.394665 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:32.394708 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.394940 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.395008 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 18:54:32.395118 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.387740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:49067)
0809 18:54:32.387880 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:32.387911 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:32.388290 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:32.388460 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:32.388463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:32.388956 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:32.389119 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.389123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.389719 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.389720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.390569 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.390572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.391029 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.391030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.391162 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.391585 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:32.391592 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:32.392771 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:32.394073 (+ 1302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:32.394076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:32.394077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:32.394256 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:32.394431 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:32.394432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:32.394432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:32.394520 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:32.394705 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:32.394708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:32.394807 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 18:54:32.394878 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 18:54:32.394981 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260809 18:54:32.395519 17279 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38087
I20260809 18:54:32.395599 17279 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:32.395751 17279 heartbeater.cc:511] Master 127.23.218.254:38087 requested a full tablet report, sending...
I20260809 18:54:32.396080 16521 ts_manager.cc:194] Registered new tserver with Master: 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459)
I20260809 18:54:32.396431 16521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:49067
I20260809 18:54:32.402082 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:54:32.408015 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.403336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.23.218.254:38087)
0809 18:54:32.403374 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.403376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.403401 (+ 25us) server_negotiation.cc:407] Connection header received
0809 18:54:32.403456 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.403456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.403464 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.403506 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:32.403982 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.404427 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.405056 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.405230 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.405269 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.406177 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:32.406178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:32.406178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:32.406184 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:32.407222 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:32.407487 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:32.407488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:32.407488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:32.407527 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:32.407642 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:32.407642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:32.407643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:32.407764 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:32.407831 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.407902 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.407959 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:54:32.409237 16521 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:54:32.410787 16521 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:54:32.416508 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.412691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34167 (local address 127.0.0.1:47344)
0809 18:54:32.412805 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:32.412845 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:32.412905 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:32.413304 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:32.413307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:32.413309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:32.413370 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.413371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.414373 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.414374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.415439 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.415441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.416236 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.416237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.416314 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.416327 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 18:54:32.416388 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 18:54:32.416439 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
I20260809 18:54:32.416527 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.412930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.23.218.194:34167)
0809 18:54:32.413113 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.413117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.413128 (+ 11us) server_negotiation.cc:407] Connection header received
0809 18:54:32.413169 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.413171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.413187 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.413286 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:32.413458 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.414258 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.415538 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.416164 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.416227 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.416281 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.416412 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.416453 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
I20260809 18:54:32.416742 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.412885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:40459 (local address 127.0.0.1:57310)
0809 18:54:32.413206 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:32.413216 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:32.413274 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:32.413631 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:32.413632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:32.413634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:32.413670 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.413671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.414841 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.414842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.415673 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.415676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.416518 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.416519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.416581 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.416586 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:54:32.416636 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 18:54:32.416674 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":66,"threads_started":1}
I20260809 18:54:32.416957 17290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.412709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:34423 (local address 127.0.0.1:45702)
0809 18:54:32.413130 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:32.413140 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:32.413183 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:32.413426 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:32.413427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:32.413430 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:32.413474 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.413475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.414798 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.414799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.415846 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.415849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.416745 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.416746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.416802 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.416805 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:54:32.416862 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 18:54:32.416897 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":88,"threads_started":1}
I20260809 18:54:32.417009 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.412965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.23.218.195:40459)
0809 18:54:32.413428 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.413432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.413442 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:54:32.413475 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.413477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.413490 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.413558 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:32.413746 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.414759 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.415740 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.416368 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.416436 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.416482 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.416908 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.416944 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":65,"threads_started":1}
I20260809 18:54:32.417018 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.413027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.23.218.193:34423)
0809 18:54:32.413173 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.413177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.413189 (+ 12us) server_negotiation.cc:407] Connection header received
0809 18:54:32.413256 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.413258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.413274 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.413344 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:32.413549 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.414691 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.415925 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.416602 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.416658 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.416703 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.416894 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.416941 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260809 18:54:32.417421 16959 tablet_service.cc:1467] Processing CreateTablet for tablet 313195e37fd34764ae7af10090701fd0 (DEFAULT_TABLE table=test-table [id=c155d5caefdc4bf8b763824f5b8a070b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:32.417533 17178 tablet_service.cc:1467] Processing CreateTablet for tablet 313195e37fd34764ae7af10090701fd0 (DEFAULT_TABLE table=test-table [id=c155d5caefdc4bf8b763824f5b8a070b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:32.417711 16959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 313195e37fd34764ae7af10090701fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:32.417850 17178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 313195e37fd34764ae7af10090701fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:32.418780 16740 tablet_service.cc:1467] Processing CreateTablet for tablet 313195e37fd34764ae7af10090701fd0 (DEFAULT_TABLE table=test-table [id=c155d5caefdc4bf8b763824f5b8a070b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:32.419077 16740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 313195e37fd34764ae7af10090701fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:32.420145 17297 tablet_bootstrap.cc:492] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: Bootstrap starting.
I20260809 18:54:32.420713 17297 tablet_bootstrap.cc:654] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:32.420739 17298 tablet_bootstrap.cc:492] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: Bootstrap starting.
I20260809 18:54:32.421317 17297 log.cc:824] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:32.421365 17298 tablet_bootstrap.cc:654] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:32.421825 17298 log.cc:824] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:32.421887 17303 tablet_bootstrap.cc:492] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Bootstrap starting.
I20260809 18:54:32.422078 17297 tablet_bootstrap.cc:492] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: No bootstrap required, opened a new log
I20260809 18:54:32.422143 17297 ts_tablet_manager.cc:1397] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:54:32.422557 17298 tablet_bootstrap.cc:492] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: No bootstrap required, opened a new log
I20260809 18:54:32.422636 17298 ts_tablet_manager.cc:1397] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:32.422959 17303 tablet_bootstrap.cc:654] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:32.423480 17303 log.cc:824] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:32.423666 17297 raft_consensus.cc:348] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:32.423834 17297 raft_consensus.cc:374] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:32.423873 17297 raft_consensus.cc:729] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931165f44627438b9bc6ebb7691ce2dd, State: Initialized, Role: FOLLOWER
I20260809 18:54:32.423983 17297 consensus_queue.cc:260] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:32.424245 17303 tablet_bootstrap.cc:492] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: No bootstrap required, opened a new log
I20260809 18:54:32.424273 17064 heartbeater.cc:503] Master 127.23.218.254:38087 was elected leader, sending a full tablet report...
I20260809 18:54:32.424300 17303 ts_tablet_manager.cc:1397] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:54:32.424437 17297 ts_tablet_manager.cc:1428] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:32.424371 17298 raft_consensus.cc:348] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:32.424516 17298 raft_consensus.cc:374] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:32.424551 17298 raft_consensus.cc:729] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59fb4a5b22a34a94ae7ce9fb171aac9a, State: Initialized, Role: FOLLOWER
I20260809 18:54:32.424665 17298 consensus_queue.cc:260] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:32.424913 17298 ts_tablet_manager.cc:1428] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:32.424999 17279 heartbeater.cc:503] Master 127.23.218.254:38087 was elected leader, sending a full tablet report...
I20260809 18:54:32.426462 17303 raft_consensus.cc:348] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:32.426594 17303 raft_consensus.cc:374] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:32.426625 17303 raft_consensus.cc:729] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7264dd200dcd4911a62df6df6f54f136, State: Initialized, Role: FOLLOWER
I20260809 18:54:32.426731 17303 consensus_queue.cc:260] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:32.426942 17303 ts_tablet_manager.cc:1428] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:54:32.426986 16845 heartbeater.cc:503] Master 127.23.218.254:38087 was elected leader, sending a full tablet report...
I20260809 18:54:32.456355 17306 raft_consensus.cc:482] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:32.456432 17306 raft_consensus.cc:504] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:32.456662 17306 leader_election.cc:288] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7264dd200dcd4911a62df6df6f54f136 (127.23.218.193:34423), 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459)
I20260809 18:54:32.459167 17066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.456791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:34423 (local address 127.23.218.194:60655)
0809 18:54:32.456837 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:32.456849 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:32.456893 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:32.457022 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:32.457023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:32.457024 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:32.457081 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.457082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.457654 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.457655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.458400 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.458402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.458938 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.458939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.459013 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.459017 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:54:32.459065 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 18:54:32.459103 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":26}
I20260809 18:54:32.459167 17296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.456858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:60655 (local address 127.23.218.193:34423)
0809 18:54:32.456910 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.456912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.456933 (+ 21us) server_negotiation.cc:407] Connection header received
0809 18:54:32.456953 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.456954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.456962 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.457004 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:32.457138 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.457584 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.458473 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.458836 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.458899 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.458907 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.459083 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.459115 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:54:32.459422 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.456953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:50715 (local address 127.23.218.195:40459)
0809 18:54:32.456990 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.456991 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.457077 (+ 86us) server_negotiation.cc:407] Connection header received
0809 18:54:32.457122 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.457123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.457130 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.457172 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:32.457294 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.457700 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.458635 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.459082 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.459167 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.459175 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.459332 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.459367 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:54:32.459400 16770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313195e37fd34764ae7af10090701fd0" candidate_uuid: "931165f44627438b9bc6ebb7691ce2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7264dd200dcd4911a62df6df6f54f136" is_pre_election: true
I20260809 18:54:32.459508 16770 raft_consensus.cc:2454] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931165f44627438b9bc6ebb7691ce2dd in term 0.
I20260809 18:54:32.459640 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.456889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:40459 (local address 127.23.218.194:50715)
0809 18:54:32.457022 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:32.457030 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:32.457077 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:32.457179 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:32.457180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:32.457181 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:32.457243 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.457244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.457765 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.457766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.458558 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:32.458560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.459178 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.459179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:32.459252 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.459256 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:54:32.459519 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 18:54:32.459559 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260809 18:54:32.459705 16877 leader_election.cc:302] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7264dd200dcd4911a62df6df6f54f136, 931165f44627438b9bc6ebb7691ce2dd; no voters:
I20260809 18:54:32.459813 17306 raft_consensus.cc:2781] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:54:32.459867 17306 raft_consensus.cc:482] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:54:32.459892 17306 raft_consensus.cc:3037] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:32.460070 17211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313195e37fd34764ae7af10090701fd0" candidate_uuid: "931165f44627438b9bc6ebb7691ce2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" is_pre_election: true
I20260809 18:54:32.460201 17211 raft_consensus.cc:2454] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931165f44627438b9bc6ebb7691ce2dd in term 0.
I20260809 18:54:32.460681 17306 raft_consensus.cc:504] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:32.460779 17306 leader_election.cc:288] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [CANDIDATE]: Term 1 election: Requested vote from peers 7264dd200dcd4911a62df6df6f54f136 (127.23.218.193:34423), 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459)
I20260809 18:54:32.460945 16770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313195e37fd34764ae7af10090701fd0" candidate_uuid: "931165f44627438b9bc6ebb7691ce2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7264dd200dcd4911a62df6df6f54f136"
I20260809 18:54:32.460954 17211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "313195e37fd34764ae7af10090701fd0" candidate_uuid: "931165f44627438b9bc6ebb7691ce2dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a"
I20260809 18:54:32.461019 17211 raft_consensus.cc:3037] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:32.461019 16770 raft_consensus.cc:3037] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:32.461756 17211 raft_consensus.cc:2454] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931165f44627438b9bc6ebb7691ce2dd in term 1.
I20260809 18:54:32.461756 16770 raft_consensus.cc:2454] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931165f44627438b9bc6ebb7691ce2dd in term 1.
I20260809 18:54:32.461899 16877 leader_election.cc:302] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7264dd200dcd4911a62df6df6f54f136, 931165f44627438b9bc6ebb7691ce2dd; no voters:
I20260809 18:54:32.461993 17306 raft_consensus.cc:2781] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:32.462112 17306 raft_consensus.cc:686] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 1 LEADER]: Becoming Leader. State: Replica: 931165f44627438b9bc6ebb7691ce2dd, State: Running, Role: LEADER
I20260809 18:54:32.462191 17306 consensus_queue.cc:237] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:32.462873 16521 catalog_manager.cc:5495] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194). New cstate: current_term: 1 leader_uuid: "931165f44627438b9bc6ebb7691ce2dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } health_report { overall_health: UNKNOWN } } }
I20260809 18:54:32.474503 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.472374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.23.218.194:34167)
0809 18:54:32.472410 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:32.472412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:32.472420 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:54:32.472465 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:32.472466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:32.472482 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:32.472534 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:54:32.472642 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.473118 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.473835 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:32.474024 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:32.474073 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:32.474216 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:54:32.474302 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:54:32.474394 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:32.474411 (+ 17us) server_negotiation.cc:299] Negotiation successful
0809 18:54:32.474428 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:54:32.475831 17211 raft_consensus.cc:1261] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 1 FOLLOWER]: Refusing update from remote peer 931165f44627438b9bc6ebb7691ce2dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:32.475852 16770 raft_consensus.cc:1261] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Refusing update from remote peer 931165f44627438b9bc6ebb7691ce2dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:32.476083 17306 consensus_queue.cc:1035] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:32.476208 17312 consensus_queue.cc:1035] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:32.478686 17321 mvcc.cc:205] Tried to move back new op lower bound from 7316691650459017216 to 7316691650405732352. Current Snapshot: MvccSnapshot[applied={T|T < 7316691650459017216 or (T in {7316691650459017216})}]
I20260809 18:54:32.479130 17322 mvcc.cc:205] Tried to move back new op lower bound from 7316691650459017216 to 7316691650405732352. Current Snapshot: MvccSnapshot[applied={T|T < 7316691650459017216}]
I20260809 18:54:32.480674 16520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:32.480747 16520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:32.481348 16520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Sending DeleteTablet for 3 replicas of tablet 313195e37fd34764ae7af10090701fd0
I20260809 18:54:32.481662 16740 tablet_service.cc:1514] Processing DeleteTablet for tablet 313195e37fd34764ae7af10090701fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:32 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:45702
I20260809 18:54:32.481666 16959 tablet_service.cc:1514] Processing DeleteTablet for tablet 313195e37fd34764ae7af10090701fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:32 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:47344
I20260809 18:54:32.481689 17178 tablet_service.cc:1514] Processing DeleteTablet for tablet 313195e37fd34764ae7af10090701fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:32 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:57310
I20260809 18:54:32.481879 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 16624
I20260809 18:54:32.492156 17335 tablet_replica.cc:330] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: stopping tablet replica
I20260809 18:54:32.492274 17335 raft_consensus.cc:2229] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:54:32.492394 17335 raft_consensus.cc:2258] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:32.492578 17339 tablet_replica.cc:330] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: stopping tablet replica
I20260809 18:54:32.492682 17339 raft_consensus.cc:2229] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:32.492777 17339 raft_consensus.cc:2258] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:32.493929 17335 ts_tablet_manager.cc:1905] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:32.495448 17335 ts_tablet_manager.cc:1918] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:54:32.495523 17335 log.cc:1196] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/313195e37fd34764ae7af10090701fd0
I20260809 18:54:32.495798 17335 ts_tablet_manager.cc:1939] T 313195e37fd34764ae7af10090701fd0 P 931165f44627438b9bc6ebb7691ce2dd: Deleting consensus metadata
I20260809 18:54:32.496212 16488 catalog_manager.cc:4841] TS 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167): tablet 313195e37fd34764ae7af10090701fd0 (table test-table [id=c155d5caefdc4bf8b763824f5b8a070b]) successfully deleted
I20260809 18:54:32.496418 17339 ts_tablet_manager.cc:1905] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:32.497861 17339 ts_tablet_manager.cc:1918] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:54:32.497928 17339 log.cc:1196] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/313195e37fd34764ae7af10090701fd0
W20260809 18:54:32.497915 16491 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:34423 (error 108)
I20260809 18:54:32.498185 17339 ts_tablet_manager.cc:1939] T 313195e37fd34764ae7af10090701fd0 P 59fb4a5b22a34a94ae7ce9fb171aac9a: Deleting consensus metadata
I20260809 18:54:32.498191 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 16852
I20260809 18:54:32.498500 17290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:32.498398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:34423 (local address 127.0.0.1:45710)
0809 18:54:32.498440 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:32.498472 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:34423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 18:54:32.498654 16491 catalog_manager.cc:4584] TS 7264dd200dcd4911a62df6df6f54f136 (127.23.218.193:34423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 313195e37fd34764ae7af10090701fd0: Network error: Client connection negotiation failed: client connection to 127.23.218.193:34423: connect: Connection refused (error 111)
I20260809 18:54:32.500305 16491 catalog_manager.cc:4841] TS 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459): tablet 313195e37fd34764ae7af10090701fd0 (table test-table [id=c155d5caefdc4bf8b763824f5b8a070b]) successfully deleted
I20260809 18:54:32.504027 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 17073
I20260809 18:54:32.511277 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 16466
I20260809 18:54:52.519423 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38087
--webserver_interface=127.23.218.254
--webserver_port=36659
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:54:52.601006 17341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:52.601238 17341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:52.601269 17341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:52.602447 17341 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 18:54:52.602484 17341 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 18:54:52.602501 17341 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 18:54:52.602515 17341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:54:52.602532 17341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:52.602556 17341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:52.602574 17341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:54:52.602597 17341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:54:52.604157 17341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:52.604403 17341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:52.604600 17341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:52.614720 17350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:52.614744 17360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:52.614768 17341 server_base.cc:1034] running on GCE node
W20260809 18:54:52.614744 17353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:52.615180 17341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:52.615473 17341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:52.616621 17341 hybrid_clock.cc:648] HybridClock initialized: now 1786301692616605 us; error 29 us; skew 500 ppm
Aug 09 18:54:52 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301692, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:52.619009 17341 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:54:52.619428 17341 webserver.cc:460] Webserver started at http://127.23.218.254:36659/ using document root <none> and password file <none>
I20260809 18:54:52.619642 17341 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:52.619706 17341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:52.621007 17341 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:52.621655 17371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:52.621822 17341 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:52.621909 17341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "9a344976f7a4471fba16c5e5e1d3ac4b"
format_stamp: "Formatted at 2026-08-09 18:54:31 on dist-test-slave-6zbq"
server_key: "c1979473a99622adb6608956619e9ba6"
server_key_iv: "8946d1807db67fbb748c761087e5a5c8"
server_key_version: "encryptionkey@0"
I20260809 18:54:52.622318 17341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:52.639012 17341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:52.640792 17341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:52.640923 17341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:52.644917 17341 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:38087
I20260809 18:54:52.644951 17466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:38087 every 8 connection(s)
I20260809 18:54:52.645318 17341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:54:52.646775 17467 sys_catalog.cc:263] Verifying existing consensus state
I20260809 18:54:52.647390 17467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Bootstrap starting.
I20260809 18:54:52.655000 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 17341
I20260809 18:54:52.655262 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:34423
--local_ip_for_outbound_sockets=127.23.218.193
--tserver_master_addrs=127.23.218.254:38087
--webserver_port=40103
--webserver_interface=127.23.218.193
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:54:52.656311 17467 log.cc:824] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:52.658057 17467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:54:52.658326 17467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Bootstrap complete.
I20260809 18:54:52.660465 17467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:54:52.660682 17467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a344976f7a4471fba16c5e5e1d3ac4b, State: Initialized, Role: FOLLOWER
I20260809 18:54:52.660822 17467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:54:52.660881 17467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:54:52.660928 17467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:54:52.660975 17467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 1 FOLLOWER]: Advancing to term 2
I20260809 18:54:52.661965 17467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:54:52.662055 17467 leader_election.cc:302] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a344976f7a4471fba16c5e5e1d3ac4b; no voters:
I20260809 18:54:52.662218 17467 leader_election.cc:288] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 18:54:52.662272 17471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 2 FOLLOWER]: Leader election won for term 2
I20260809 18:54:52.662429 17471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 2 LEADER]: Becoming Leader. State: Replica: 9a344976f7a4471fba16c5e5e1d3ac4b, State: Running, Role: LEADER
I20260809 18:54:52.662492 17467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:54:52.662545 17471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:54:52.662839 17471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a344976f7a4471fba16c5e5e1d3ac4b. Latest consensus state: current_term: 2 leader_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } } }
I20260809 18:54:52.662989 17471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:52.663194 17480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } } }
I20260809 18:54:52.663338 17480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: This master's current role is: LEADER
I20260809 18:54:52.663350 17489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:54:52.664228 17489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:54:52.664335 17489 catalog_manager.cc:1259] Loaded cluster ID: 4bcc72dfe4674485a6b367a9b779b96f
I20260809 18:54:52.664369 17489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:54:52.665552 17489 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:54:52.665748 17489 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Loaded TSK: 0
I20260809 18:54:52.677742 17489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Generated new TSK 1
I20260809 18:54:52.677826 17489 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 18:54:52.736888 17469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:52.737074 17469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:52.737104 17469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:52.738180 17469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:52.738221 17469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:52.738256 17469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:54:52.739671 17469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:34423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=40103
--tserver_master_addrs=127.23.218.254:38087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:52.739887 17469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:52.740098 17469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:52.744714 17508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:52.744697 17513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:52.744840 17510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:52.745456 17469 server_base.cc:1034] running on GCE node
I20260809 18:54:52.745644 17469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:52.745854 17469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:52.746987 17469 hybrid_clock.cc:648] HybridClock initialized: now 1786301692746975 us; error 30 us; skew 500 ppm
Aug 09 18:54:52 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301692, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:52.749406 17469 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:54:52.749894 17469 webserver.cc:460] Webserver started at http://127.23.218.193:40103/ using document root <none> and password file <none>
I20260809 18:54:52.750124 17469 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:52.750185 17469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:52.751487 17469 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:52.752125 17526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:52.752300 17469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:52.752403 17469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "7264dd200dcd4911a62df6df6f54f136"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "80b235e873a24e8197f885088cd93693"
server_key_iv: "5227b330ec70860f90235207d78b81b9"
server_key_version: "encryptionkey@0"
I20260809 18:54:52.752822 17469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:52.773092 17469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:52.775282 17469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:52.775429 17469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:52.775665 17469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:52.776165 17539 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 18:54:52.777514 17469 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 18:54:52.777572 17469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 18:54:52.777601 17469 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 18:54:52.778435 17469 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 18:54:52.778476 17469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:52.778535 17539 tablet_bootstrap.cc:492] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Bootstrap starting.
I20260809 18:54:52.785207 17469 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:34423
I20260809 18:54:52.785358 17729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:34423 every 8 connection(s)
I20260809 18:54:52.785609 17469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:54:52.788822 17539 log.cc:824] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:52.790596 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 17469
I20260809 18:54:52.790890 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:34167
--local_ip_for_outbound_sockets=127.23.218.194
--tserver_master_addrs=127.23.218.254:38087
--webserver_port=45263
--webserver_interface=127.23.218.194
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:54:52.791375 17539 tablet_bootstrap.cc:492] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:54:52.791713 17539 tablet_bootstrap.cc:492] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Bootstrap complete.
I20260809 18:54:52.792918 17539 ts_tablet_manager.cc:1397] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.006s
Aug 09 18:54:52 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301692, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:52.794215 17539 raft_consensus.cc:348] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:52.794552 17539 raft_consensus.cc:729] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7264dd200dcd4911a62df6df6f54f136, State: Initialized, Role: FOLLOWER
I20260809 18:54:52.794710 17539 consensus_queue.cc:260] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } } peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } }
I20260809 18:54:52.794975 17539 ts_tablet_manager.cc:1428] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:52.798372 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:52.786417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:57177 (local address 127.23.218.254:38087)
0809 18:54:52.786624 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:52.786634 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:52.786897 (+ 263us) server_negotiation.cc:407] Connection header received
0809 18:54:52.787658 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:52.787667 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:52.787744 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:52.787887 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:52.788760 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.790092 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.791551 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.792136 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.792656 (+ 520us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:52.794727 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:52.794732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:52.794732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:52.794745 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:52.796597 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:52.797095 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:52.797099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:52.797099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:52.797208 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:52.797461 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:52.797462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:52.797463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:52.797674 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:52.797753 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:52.798043 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 18:54:52.798164 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260809 18:54:52.798378 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:52.786297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:57177)
0809 18:54:52.786723 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:52.786772 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:52.787420 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:52.787962 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:52.787966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:52.788357 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:52.788626 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:52.788634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.790209 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.790210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:52.791452 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:52.791455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.792253 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.792254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:52.792381 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:52.792949 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:52.792958 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:52.794627 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:52.796704 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:52.796712 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:52.796714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:52.797006 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:52.797286 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:52.797287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:52.797287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:52.797395 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:52.797751 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:52.797759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:52.797883 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 18:54:52.797964 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 18:54:52.798135 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":126,"threads_started":1}
I20260809 18:54:52.799161 17731 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38087
I20260809 18:54:52.799256 17731 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:52.799424 17731 heartbeater.cc:511] Master 127.23.218.254:38087 requested a full tablet report, sending...
I20260809 18:54:52.800127 17407 ts_manager.cc:194] Registered new tserver with Master: 7264dd200dcd4911a62df6df6f54f136 (127.23.218.193:34423)
I20260809 18:54:52.801322 17407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:57177
I20260809 18:54:52.802258 17731 heartbeater.cc:503] Master 127.23.218.254:38087 was elected leader, sending a full tablet report...
Aug 09 18:54:52 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301692, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.193@KRBTEST.COM
I20260809 18:54:52.808822 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:52.800742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:34423 (local address 127.0.0.1:46752)
0809 18:54:52.801002 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:52.801027 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:52.801107 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:52.801263 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:52.801268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:52.801561 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:52.801617 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:52.801620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.802790 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.802791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:52.803704 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:52.803708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.804126 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.804127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:52.804207 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:52.804571 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:52.804575 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:52.806063 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:52.807921 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:52.807928 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:52.807929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:52.808107 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:52.808288 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:52.808289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:52.808289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:52.808335 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:52.808582 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:52.808586 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:52.808622 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 18:54:52.808677 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 18:54:52.808762 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":82,"threads_started":1}
I20260809 18:54:52.808902 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:52.800804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46752 (local address 127.23.218.193:34423)
0809 18:54:52.800944 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:52.800949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:52.801099 (+ 150us) server_negotiation.cc:407] Connection header received
0809 18:54:52.801184 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:52.801186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:52.801200 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:52.801255 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:52.801702 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.802726 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.803777 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.804063 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.804114 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:52.806138 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:52.806146 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:52.806146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:52.806152 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:52.807849 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:52.808171 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:52.808173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:52.808174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:52.808237 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:52.808381 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:52.808382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:52.808382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:52.808533 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:52.808603 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:52.808714 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 18:54:52.808805 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260809 18:54:52.809198 17620 tablet_service.cc:1514] Processing DeleteTablet for tablet 313195e37fd34764ae7af10090701fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:32 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:46752
I20260809 18:54:52.809686 17743 tablet_replica.cc:330] stopping tablet replica
I20260809 18:54:52.809772 17743 raft_consensus.cc:2229] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:52.809835 17743 raft_consensus.cc:2258] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:52.810087 17743 ts_tablet_manager.cc:1905] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:52.811910 17743 ts_tablet_manager.cc:1918] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:54:52.811968 17743 log.cc:1196] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal/wals/313195e37fd34764ae7af10090701fd0
I20260809 18:54:52.812163 17743 ts_tablet_manager.cc:1939] T 313195e37fd34764ae7af10090701fd0 P 7264dd200dcd4911a62df6df6f54f136: Deleting consensus metadata
I20260809 18:54:52.812594 17380 catalog_manager.cc:4841] TS 7264dd200dcd4911a62df6df6f54f136 (127.23.218.193:34423): tablet 313195e37fd34764ae7af10090701fd0 (table test-table [id=c155d5caefdc4bf8b763824f5b8a070b]) successfully deleted
W20260809 18:54:52.878592 17736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:52.878780 17736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:52.878809 17736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:52.879904 17736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:52.879942 17736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:52.879976 17736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:54:52.881402 17736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:34167
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=45263
--tserver_master_addrs=127.23.218.254:38087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:52.881608 17736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:52.881805 17736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:54:52.886427 17752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:52.886489 17754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:52.886436 17749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:52.886520 17736 server_base.cc:1034] running on GCE node
I20260809 18:54:52.886834 17736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:52.887046 17736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:52.888183 17736 hybrid_clock.cc:648] HybridClock initialized: now 1786301692888170 us; error 32 us; skew 500 ppm
Aug 09 18:54:52 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301692, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:52.890432 17736 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:54:52.890818 17736 webserver.cc:460] Webserver started at http://127.23.218.194:45263/ using document root <none> and password file <none>
I20260809 18:54:52.891002 17736 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:52.891057 17736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:52.892279 17736 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:52.892798 17764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:52.892946 17736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:54:52.893025 17736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "931165f44627438b9bc6ebb7691ce2dd"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "c99ac5d2bb1460c6c22e4ff6e6f89e7e"
server_key_iv: "9b7389e0a235c1d6dd9151dc3408dc6e"
server_key_version: "encryptionkey@0"
I20260809 18:54:52.893330 17736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:52.905579 17736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:52.907151 17736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:52.907277 17736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:52.907460 17736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:52.907722 17736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:52.907754 17736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:52.907779 17736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:52.907801 17736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:52.913091 17736 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:34167
I20260809 18:54:52.913149 17961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:34167 every 8 connection(s)
I20260809 18:54:52.913429 17736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:54:52.918255 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 17736
I20260809 18:54:52.918537 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:40459
--local_ip_for_outbound_sockets=127.23.218.195
--tserver_master_addrs=127.23.218.254:38087
--webserver_port=43867
--webserver_interface=127.23.218.195
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 18:54:52 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301692, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:52.921681 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:52.914029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:35821 (local address 127.23.218.254:38087)
0809 18:54:52.914092 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:52.914093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:52.914287 (+ 194us) server_negotiation.cc:407] Connection header received
0809 18:54:52.914739 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:52.914741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:52.914753 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:52.914803 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:52.915615 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.916096 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.916986 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.917177 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.917221 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:52.919220 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:52.919222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:52.919222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:52.919230 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:52.920438 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:52.920855 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:52.920856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:52.920856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:52.920899 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:52.921148 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:52.921148 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:52.921148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:52.921246 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:52.921289 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:52.921555 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 18:54:52.921621 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 18:54:52.921945 17965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:52.914017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:35821)
0809 18:54:52.914162 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:52.914211 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:52.914660 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:52.914853 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:52.914856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:52.915378 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:52.915551 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:52.915555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.916155 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.916155 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:52.916928 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:52.916931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:52.917230 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:52.917230 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:52.917328 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:52.917775 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:52.917783 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:52.919127 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:52.920513 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:52.920521 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:52.920523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:52.920789 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:52.920951 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:52.920952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:52.920952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:52.921091 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:52.921299 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:52.921305 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:52.921443 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 18:54:52.921546 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 18:54:52.921691 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260809 18:54:52.922477 17962 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38087
I20260809 18:54:52.922585 17962 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:52.922772 17962 heartbeater.cc:511] Master 127.23.218.254:38087 requested a full tablet report, sending...
I20260809 18:54:52.923163 17407 ts_manager.cc:194] Registered new tserver with Master: 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
I20260809 18:54:52.923514 17407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:35821
W20260809 18:54:53.000105 17968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:54:53.000290 17968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:54:53.000312 17968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:54:53.001468 17968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:54:53.001510 17968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:54:53.001538 17968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:54:53.003074 17968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:40459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=43867
--tserver_master_addrs=127.23.218.254:38087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:54:53.003291 17968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:54:53.003499 17968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 18:54:53.008136 17968 server_base.cc:1034] running on GCE node
W20260809 18:54:53.008080 17978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:53.008070 17982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:54:53.008100 17980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:54:53.008515 17968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:54:53.008744 17968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:54:53.009881 17968 hybrid_clock.cc:648] HybridClock initialized: now 1786301693009861 us; error 36 us; skew 500 ppm
Aug 09 18:54:53 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301693, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:54:53.012262 17968 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:54:53.012655 17968 webserver.cc:460] Webserver started at http://127.23.218.195:43867/ using document root <none> and password file <none>
I20260809 18:54:53.012838 17968 fs_manager.cc:359] Metadata directory not provided
I20260809 18:54:53.012895 17968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:54:53.014117 17968 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:54:53.014690 17989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:53.014835 17968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:53.014917 17968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "11af8815e8c800ff2a8bedab56aa38f3"
server_key_iv: "50f927fa48cc69c9fb5bc27a92c205fe"
server_key_version: "encryptionkey@0"
I20260809 18:54:53.015228 17968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:54:53.035521 17968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:54:53.037503 17968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:54:53.037631 17968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:54:53.037832 17968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:54:53.038102 17968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:54:53.038136 17968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:53.038162 17968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:54:53.038179 17968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:54:53.043543 17968 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:40459
I20260809 18:54:53.043581 18189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:40459 every 8 connection(s)
I20260809 18:54:53.043885 17968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:54:53.044211 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 17968
Aug 09 18:54:53 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301693, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:54:53.051537 18201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.044634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47082 (local address 127.23.218.254:38087)
0809 18:54:53.044729 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:53.044732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:53.044836 (+ 104us) server_negotiation.cc:407] Connection header received
0809 18:54:53.044925 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:53.044927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:53.044941 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:53.045007 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:53.045573 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.046372 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.047390 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.047677 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.047756 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.048935 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:53.048936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:53.048936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:53.048944 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:53.050455 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:53.050783 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:53.050784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:53.050784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:53.050846 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:53.051054 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:53.051054 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:53.051055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:53.051144 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:53.051311 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:53.051353 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 18:54:53.051432 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":42,"threads_started":1}
I20260809 18:54:53.053462 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.044605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:53539 (local address 127.23.218.254:38087)
0809 18:54:53.044667 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:53.044669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:53.044866 (+ 197us) server_negotiation.cc:407] Connection header received
0809 18:54:53.045414 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:53.045415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:53.045425 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:53.045497 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:53.046627 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.047346 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.048524 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.048810 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.048884 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.051183 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:53.051184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:53.051184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:53.051191 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:53.052505 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:53.052838 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:53.052838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:53.052839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:53.052879 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:53.053043 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:53.053044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:53.053044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:53.053123 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:53.053165 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:53.053356 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 18:54:53.053408 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 18:54:53.053604 18202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.044596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:53539)
0809 18:54:53.044758 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:53.044796 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:53.045320 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:53.045557 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:53.045562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:53.046259 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:54:53.046535 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.046541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.047427 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.047428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.048443 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.048445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.048884 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.048885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.049019 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.049547 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:54:53.049557 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:54:53.051110 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:54:53.052569 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:53.052576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:53.052577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:53.052784 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:53.052920 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:54:53.052920 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:54:53.052920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:54:53.053004 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:54:53.053162 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:54:53.053165 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:54:53.053266 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 18:54:53.053339 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 18:54:53.053443 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260809 18:54:53.054023 18192 heartbeater.cc:347] Connected to a master server at 127.23.218.254:38087
I20260809 18:54:53.054100 18192 heartbeater.cc:464] Registering TS with master...
I20260809 18:54:53.054239 18192 heartbeater.cc:511] Master 127.23.218.254:38087 requested a full tablet report, sending...
I20260809 18:54:53.054554 17407 ts_manager.cc:194] Registered new tserver with Master: 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459)
I20260809 18:54:53.054860 17407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:53539
I20260809 18:54:53.062938 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:54:53.065183 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.063261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47094 (local address 127.23.218.254:38087)
0809 18:54:53.063310 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:53.063312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:53.063421 (+ 109us) server_negotiation.cc:407] Connection header received
0809 18:54:53.063493 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:53.063493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:53.063522 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:53.063573 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:54:53.063640 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.064055 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.064724 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.064885 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.064925 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.065021 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:54:53.065055 (+ 34us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 18:54:53.065144 (+ 89us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47094: token expired
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 18:54:53.065230 17734 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47094: token expired
W20260809 18:54:53.065409 17284 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.23.218.254:38087)
I20260809 18:54:53.070305 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.065589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47104 (local address 127.23.218.254:38087)
0809 18:54:53.065639 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:53.065641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:53.065662 (+ 21us) server_negotiation.cc:407] Connection header received
0809 18:54:53.065680 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:53.065681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:53.065689 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:53.065731 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:54:53.066128 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.066584 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.067207 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.067379 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.067428 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.068378 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:54:53.068379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:54:53.068379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:54:53.068386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:54:53.069397 (+ 1011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:53.069673 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:53.069674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:53.069674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:53.069731 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:54:53.069888 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:54:53.069888 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:54:53.069889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:54:53.069985 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:54:53.070064 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:53.070162 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 18:54:53.070230 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:54:53.071239 17407 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:54:53.072782 17407 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:54:53.074717 17620 tablet_service.cc:1467] Processing CreateTablet for tablet e7f03ddf09ef4bdd8ac446aff953310a (DEFAULT_TABLE table=test-table [id=58e7865c21494acaa6c46ed1080341eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:53.074927 17620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f03ddf09ef4bdd8ac446aff953310a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:53.076556 17539 tablet_bootstrap.cc:492] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Bootstrap starting.
I20260809 18:54:53.076987 17539 tablet_bootstrap.cc:654] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:53.077539 17539 tablet_bootstrap.cc:492] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: No bootstrap required, opened a new log
I20260809 18:54:53.077586 17539 ts_tablet_manager.cc:1397] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:53.077921 17539 raft_consensus.cc:348] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:54:53.077975 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.074398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:40459 (local address 127.0.0.1:59956)
0809 18:54:53.074428 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:53.074438 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:53.074476 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:53.074822 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:53.074822 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:53.074824 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:53.074863 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.074865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.075980 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.075981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.076930 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.076933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.077734 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.077735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.077803 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.077823 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 18:54:53.077869 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 18:54:53.077914 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 18:54:53.078004 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.074478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.23.218.195:40459)
0809 18:54:53.074664 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:53.074668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:53.074677 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:54:53.074716 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:53.074717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:53.074731 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:53.074811 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:53.074930 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.075889 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.077017 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.077658 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.077719 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.077772 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:53.077901 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 18:54:53.077935 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260809 18:54:53.077991 17539 raft_consensus.cc:374] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:53.078037 17539 raft_consensus.cc:729] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7264dd200dcd4911a62df6df6f54f136, State: Initialized, Role: FOLLOWER
I20260809 18:54:53.078086 17539 consensus_queue.cc:260] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:54:53.078179 17539 ts_tablet_manager.cc:1428] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 18:54:53.078667 18072 tablet_service.cc:1467] Processing CreateTablet for tablet e7f03ddf09ef4bdd8ac446aff953310a (DEFAULT_TABLE table=test-table [id=58e7865c21494acaa6c46ed1080341eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:53.078958 18072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f03ddf09ef4bdd8ac446aff953310a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:53.081595 18212 tablet_bootstrap.cc:492] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: Bootstrap starting.
I20260809 18:54:53.081737 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.074401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34167 (local address 127.0.0.1:59788)
0809 18:54:53.074774 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:53.074783 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:53.074949 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:53.075128 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:53.075129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:53.075131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:53.075169 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.075170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.076184 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.076185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.077171 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.077175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.078137 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.078139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.078208 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.078211 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:54:53.081608 (+ 3397us) client_negotiation.cc:240] Negotiation successful
0809 18:54:53.081658 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":242,"thread_start_us":70,"threads_started":1}
I20260809 18:54:53.081789 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.074520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.23.218.194:34167)
0809 18:54:53.074715 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:53.074719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:53.074872 (+ 153us) server_negotiation.cc:407] Connection header received
0809 18:54:53.075028 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:53.075031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:53.075047 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:53.075120 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:53.075229 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.076102 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.077303 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.077990 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.078063 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.078118 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:53.081653 (+ 3535us) server_negotiation.cc:299] Negotiation successful
0809 18:54:53.081711 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260809 18:54:53.082314 18212 tablet_bootstrap.cc:654] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:53.082520 17841 tablet_service.cc:1467] Processing CreateTablet for tablet e7f03ddf09ef4bdd8ac446aff953310a (DEFAULT_TABLE table=test-table [id=58e7865c21494acaa6c46ed1080341eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:54:53.082875 17841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f03ddf09ef4bdd8ac446aff953310a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:54:53.082898 18212 log.cc:824] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:53.083683 18212 tablet_bootstrap.cc:492] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: No bootstrap required, opened a new log
I20260809 18:54:53.083742 18212 ts_tablet_manager.cc:1397] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:53.085598 18212 raft_consensus.cc:348] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:54:53.085767 18212 raft_consensus.cc:374] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:53.085803 18212 raft_consensus.cc:729] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59fb4a5b22a34a94ae7ce9fb171aac9a, State: Initialized, Role: FOLLOWER
I20260809 18:54:53.085935 18212 consensus_queue.cc:260] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:54:53.086210 18212 ts_tablet_manager.cc:1428] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 18:54:53.086236 18192 heartbeater.cc:503] Master 127.23.218.254:38087 was elected leader, sending a full tablet report...
I20260809 18:54:53.086426 18216 tablet_bootstrap.cc:492] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: Bootstrap starting.
I20260809 18:54:53.086982 18216 tablet_bootstrap.cc:654] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: Neither blocks nor log segments found. Creating new log.
I20260809 18:54:53.087441 18216 log.cc:824] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: Log is configured to *not* fsync() on all Append() calls
I20260809 18:54:53.088044 18216 tablet_bootstrap.cc:492] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: No bootstrap required, opened a new log
I20260809 18:54:53.088102 18216 ts_tablet_manager.cc:1397] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:54:53.089370 18216 raft_consensus.cc:348] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:54:53.089496 18216 raft_consensus.cc:374] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:54:53.089529 18216 raft_consensus.cc:729] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931165f44627438b9bc6ebb7691ce2dd, State: Initialized, Role: FOLLOWER
I20260809 18:54:53.089635 18216 consensus_queue.cc:260] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:54:53.089875 18216 ts_tablet_manager.cc:1428] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:54:53.089911 17962 heartbeater.cc:503] Master 127.23.218.254:38087 was elected leader, sending a full tablet report...
W20260809 18:54:53.163923 17963 tablet.cc:2369] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:54:53.231760 17739 raft_consensus.cc:482] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:54:53.231887 17739 raft_consensus.cc:504] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:54:53.232227 17739 leader_election.cc:288] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459), 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
I20260809 18:54:53.235047 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.232477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:36797 (local address 127.23.218.195:40459)
0809 18:54:53.232515 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:53.232517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:53.232526 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:54:53.232579 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:53.232579 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:53.232590 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:53.232643 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:53.232821 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.233311 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.234308 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.234712 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.234769 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.234778 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:53.234950 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 18:54:53.234984 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:54:53.235061 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.232374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:40459 (local address 127.23.218.193:36797)
0809 18:54:53.232433 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:53.232443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:53.232502 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:53.232656 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:53.232657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:53.232658 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:53.232750 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.232751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.233397 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.233399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.234217 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.234219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.234783 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.234784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.234865 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.234883 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 18:54:53.234932 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 18:54:53.234977 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 18:54:53.235054 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.232442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:39789 (local address 127.23.218.194:34167)
0809 18:54:53.232489 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:53.232490 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:53.232651 (+ 161us) server_negotiation.cc:407] Connection header received
0809 18:54:53.232684 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:53.232685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:53.232695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:53.232751 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:54:53.232928 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.233437 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.234327 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.234731 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.234798 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.234806 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:53.234952 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 18:54:53.234989 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:54:53.235064 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.232385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34167 (local address 127.23.218.193:39789)
0809 18:54:53.232565 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:53.232575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:54:53.232642 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:54:53.232757 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:54:53.232758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:54:53.232759 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:54:53.232837 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.232838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.233517 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.233517 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.234240 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:54:53.234242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.234830 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.234832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:54:53.234882 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.234886 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:54:53.234934 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 18:54:53.234972 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
I20260809 18:54:53.235435 18102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f03ddf09ef4bdd8ac446aff953310a" candidate_uuid: "7264dd200dcd4911a62df6df6f54f136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" is_pre_election: true
I20260809 18:54:53.235606 18102 raft_consensus.cc:2454] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7264dd200dcd4911a62df6df6f54f136 in term 0.
I20260809 18:54:53.235666 17878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f03ddf09ef4bdd8ac446aff953310a" candidate_uuid: "7264dd200dcd4911a62df6df6f54f136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931165f44627438b9bc6ebb7691ce2dd" is_pre_election: true
I20260809 18:54:53.235831 17878 raft_consensus.cc:2454] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7264dd200dcd4911a62df6df6f54f136 in term 0.
I20260809 18:54:53.235841 17531 leader_election.cc:302] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59fb4a5b22a34a94ae7ce9fb171aac9a, 7264dd200dcd4911a62df6df6f54f136; no voters:
I20260809 18:54:53.235957 17739 raft_consensus.cc:2781] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:54:53.236034 17739 raft_consensus.cc:482] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:54:53.236059 17739 raft_consensus.cc:3037] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:53.236732 17739 raft_consensus.cc:504] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:54:53.236852 17739 leader_election.cc:288] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 1 election: Requested vote from peers 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459), 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
I20260809 18:54:53.237006 17878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f03ddf09ef4bdd8ac446aff953310a" candidate_uuid: "7264dd200dcd4911a62df6df6f54f136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "931165f44627438b9bc6ebb7691ce2dd"
I20260809 18:54:53.237082 17878 raft_consensus.cc:3037] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:53.237748 17878 raft_consensus.cc:2454] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7264dd200dcd4911a62df6df6f54f136 in term 1.
I20260809 18:54:53.237847 18102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f03ddf09ef4bdd8ac446aff953310a" candidate_uuid: "7264dd200dcd4911a62df6df6f54f136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a"
I20260809 18:54:53.237931 17529 leader_election.cc:302] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7264dd200dcd4911a62df6df6f54f136, 931165f44627438b9bc6ebb7691ce2dd; no voters:
I20260809 18:54:53.237926 18102 raft_consensus.cc:3037] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:54:53.238008 17739 raft_consensus.cc:2781] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:54:53.238164 17739 raft_consensus.cc:686] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 LEADER]: Becoming Leader. State: Replica: 7264dd200dcd4911a62df6df6f54f136, State: Running, Role: LEADER
I20260809 18:54:53.238237 17739 consensus_queue.cc:237] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:54:53.238818 18102 raft_consensus.cc:2454] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7264dd200dcd4911a62df6df6f54f136 in term 1.
I20260809 18:54:53.239063 17407 catalog_manager.cc:5495] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7264dd200dcd4911a62df6df6f54f136 (127.23.218.193). New cstate: current_term: 1 leader_uuid: "7264dd200dcd4911a62df6df6f54f136" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } health_report { overall_health: HEALTHY } } }
I20260809 18:54:53.256568 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.254609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46760 (local address 127.23.218.193:34423)
0809 18:54:53.254656 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:54:53.254658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:54:53.254677 (+ 19us) server_negotiation.cc:407] Connection header received
0809 18:54:53.254701 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:54:53.254702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:54:53.254717 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:54:53.254764 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:54:53.254855 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.255300 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.255950 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:54:53.256166 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:54:53.256223 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:54:53.256332 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:54:53.256418 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:54:53.256462 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:54:53.256477 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 18:54:53.256507 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 18:54:53.257848 18102 raft_consensus.cc:1261] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 1 FOLLOWER]: Refusing update from remote peer 7264dd200dcd4911a62df6df6f54f136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:53.257856 17878 raft_consensus.cc:1261] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [term 1 FOLLOWER]: Refusing update from remote peer 7264dd200dcd4911a62df6df6f54f136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:54:53.258173 18222 consensus_queue.cc:1035] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:53.258303 17739 consensus_queue.cc:1035] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:54:53.262519 18229 mvcc.cc:205] Tried to move back new op lower bound from 7316691735582187520 to 7316691735504424960. Current Snapshot: MvccSnapshot[applied={T|T < 7316691735582187520 or (T in {7316691735582187520})}]
I20260809 18:54:53.262769 17404 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:53.262853 17404 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:54:53.263491 17404 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Sending DeleteTablet for 3 replicas of tablet e7f03ddf09ef4bdd8ac446aff953310a
I20260809 18:54:53.263821 17841 tablet_service.cc:1514] Processing DeleteTablet for tablet e7f03ddf09ef4bdd8ac446aff953310a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:53 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:59788
I20260809 18:54:53.263885 17620 tablet_service.cc:1514] Processing DeleteTablet for tablet e7f03ddf09ef4bdd8ac446aff953310a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:53 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:46752
I20260809 18:54:53.263924 18072 tablet_service.cc:1514] Processing DeleteTablet for tablet e7f03ddf09ef4bdd8ac446aff953310a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:54:53 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:59956
I20260809 18:54:53.264014 17743 tablet_replica.cc:330] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: stopping tablet replica
I20260809 18:54:53.264091 17743 raft_consensus.cc:2229] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:54:53.264161 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 17469
I20260809 18:54:53.264178 17743 raft_consensus.cc:2258] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:53.264253 18239 tablet_replica.cc:330] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: stopping tablet replica
I20260809 18:54:53.264393 18239 raft_consensus.cc:2229] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:53.264544 18239 raft_consensus.cc:2258] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:53.271808 18239 ts_tablet_manager.cc:1905] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:53.273319 18239 ts_tablet_manager.cc:1918] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:54:53.273394 18239 log.cc:1196] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/e7f03ddf09ef4bdd8ac446aff953310a
I20260809 18:54:53.273769 18239 ts_tablet_manager.cc:1939] T e7f03ddf09ef4bdd8ac446aff953310a P 59fb4a5b22a34a94ae7ce9fb171aac9a: Deleting consensus metadata
I20260809 18:54:53.274715 17380 catalog_manager.cc:4841] TS 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459): tablet e7f03ddf09ef4bdd8ac446aff953310a (table test-table [id=58e7865c21494acaa6c46ed1080341eb]) successfully deleted
I20260809 18:54:53.276381 18238 tablet_replica.cc:330] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: stopping tablet replica
I20260809 18:54:53.276466 18238 raft_consensus.cc:2229] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:54:53.276542 18238 raft_consensus.cc:2258] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:54:53.277165 18238 ts_tablet_manager.cc:1905] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:54:53.278415 18238 ts_tablet_manager.cc:1918] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:54:53.278481 18238 log.cc:1196] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/e7f03ddf09ef4bdd8ac446aff953310a
I20260809 18:54:53.278690 18238 ts_tablet_manager.cc:1939] T e7f03ddf09ef4bdd8ac446aff953310a P 931165f44627438b9bc6ebb7691ce2dd: Deleting consensus metadata
I20260809 18:54:53.279053 17372 catalog_manager.cc:4841] TS 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167): tablet e7f03ddf09ef4bdd8ac446aff953310a (table test-table [id=58e7865c21494acaa6c46ed1080341eb]) successfully deleted
W20260809 18:54:53.280860 17380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:34423 (error 108)
I20260809 18:54:53.281255 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 17736
I20260809 18:54:53.281315 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:54:53.281201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:34423 (local address 127.0.0.1:46764)
0809 18:54:53.281242 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 18:54:53.281282 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:34423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 18:54:53.300388 17380 catalog_manager.cc:4584] TS 7264dd200dcd4911a62df6df6f54f136 (127.23.218.193:34423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7f03ddf09ef4bdd8ac446aff953310a: Network error: Client connection negotiation failed: client connection to 127.23.218.193:34423: connect: Connection refused (error 111)
I20260809 18:54:53.304136 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 17968
I20260809 18:54:53.325316 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 17341
I20260809 18:55:13.334112 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38087
--webserver_interface=127.23.218.254
--webserver_port=36659
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:13.416414 18242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:13.416590 18242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:13.416610 18242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:13.417695 18242 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 18:55:13.417732 18242 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 18:55:13.417743 18242 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 18:55:13.417764 18242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:13.417788 18242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:13.417810 18242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:13.417833 18242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:13.417856 18242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:13.419334 18242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=36659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:13.419559 18242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:13.419750 18242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:13.424404 18249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:13.424419 18252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:13.424597 18242 server_base.cc:1034] running on GCE node
W20260809 18:55:13.424454 18248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:13.424865 18242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:13.425137 18242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:13.426267 18242 hybrid_clock.cc:648] HybridClock initialized: now 1786301713426255 us; error 32 us; skew 500 ppm
Aug 09 18:55:13 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:13.428525 18242 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:55:13.428925 18242 webserver.cc:460] Webserver started at http://127.23.218.254:36659/ using document root <none> and password file <none>
I20260809 18:55:13.429105 18242 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:13.429142 18242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:13.430296 18242 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:55:13.430902 18266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.431046 18242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.431123 18242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "9a344976f7a4471fba16c5e5e1d3ac4b"
format_stamp: "Formatted at 2026-08-09 18:54:31 on dist-test-slave-6zbq"
server_key: "c1979473a99622adb6608956619e9ba6"
server_key_iv: "8946d1807db67fbb748c761087e5a5c8"
server_key_version: "encryptionkey@0"
I20260809 18:55:13.431409 18242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:13.446674 18242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:13.448179 18242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:13.448294 18242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:13.452066 18242 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:38087
I20260809 18:55:13.452102 18343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:38087 every 8 connection(s)
I20260809 18:55:13.452425 18242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:13.453742 18344 sys_catalog.cc:263] Verifying existing consensus state
I20260809 18:55:13.454412 18344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Bootstrap starting.
I20260809 18:55:13.459743 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 18242
I20260809 18:55:13.460068 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:34423
--local_ip_for_outbound_sockets=127.23.218.193
--tserver_master_addrs=127.23.218.254:38087
--webserver_port=40103
--webserver_interface=127.23.218.193
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:55:13.463778 18344 log.cc:824] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:13.465731 18344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:55:13.465987 18344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Bootstrap complete.
I20260809 18:55:13.468008 18344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:55:13.468223 18344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a344976f7a4471fba16c5e5e1d3ac4b, State: Initialized, Role: FOLLOWER
I20260809 18:55:13.468357 18344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:55:13.468415 18344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:13.468454 18344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:13.468498 18344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 2 FOLLOWER]: Advancing to term 3
I20260809 18:55:13.469491 18344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:55:13.469578 18344 leader_election.cc:302] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a344976f7a4471fba16c5e5e1d3ac4b; no voters:
I20260809 18:55:13.469719 18344 leader_election.cc:288] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 18:55:13.469766 18351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 3 FOLLOWER]: Leader election won for term 3
I20260809 18:55:13.469919 18351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [term 3 LEADER]: Becoming Leader. State: Replica: 9a344976f7a4471fba16c5e5e1d3ac4b, State: Running, Role: LEADER
I20260809 18:55:13.469972 18344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:13.470095 18351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } }
I20260809 18:55:13.470458 18356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a344976f7a4471fba16c5e5e1d3ac4b. Latest consensus state: current_term: 3 leader_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } } }
I20260809 18:55:13.470466 18354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a344976f7a4471fba16c5e5e1d3ac4b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38087 } } }
I20260809 18:55:13.470558 18356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:13.470573 18354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:13.470826 18368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:13.471549 18368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:13.471707 18368 catalog_manager.cc:1259] Loaded cluster ID: 4bcc72dfe4674485a6b367a9b779b96f
I20260809 18:55:13.471910 18368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:13.473064 18368 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:13.473242 18368 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Loaded TSK: 0
I20260809 18:55:13.473294 18368 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a344976f7a4471fba16c5e5e1d3ac4b: Loaded TSK: 1
F20260809 18:55:13.473510 18368 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f442a6299ad google::LogMessage::Fail() at ??:0
@ 0x7f442a62c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f442a6294de google::LogMessage::Flush() at ??:0
@ 0x7f442a62aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f442cd96104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f442cd93d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f44325831cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f443258348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4432583594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f4432549c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f4432553ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f442c4f70d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f442c4ec954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f442d3506db start_thread at ??:0
@ 0x7f442912761f clone at ??:0
W20260809 18:55:13.556799 18348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:13.556970 18348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:13.556993 18348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:13.558061 18348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:13.558101 18348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:13.558125 18348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:55:13.559679 18348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:34423
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=40103
--tserver_master_addrs=127.23.218.254:38087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:13.559880 18348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:13.560093 18348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:13.565376 18419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:13.565399 18415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:13.565510 18417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:13.565534 18348 server_base.cc:1034] running on GCE node
I20260809 18:55:13.565734 18348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:13.565946 18348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:13.567086 18348 hybrid_clock.cc:648] HybridClock initialized: now 1786301713567075 us; error 28 us; skew 500 ppm
Aug 09 18:55:13 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:13.569377 18348 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:55:13.569772 18348 webserver.cc:460] Webserver started at http://127.23.218.193:40103/ using document root <none> and password file <none>
I20260809 18:55:13.569976 18348 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:13.570019 18348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:13.571252 18348 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:55:13.571854 18434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.572012 18348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:13.572080 18348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "7264dd200dcd4911a62df6df6f54f136"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "80b235e873a24e8197f885088cd93693"
server_key_iv: "5227b330ec70860f90235207d78b81b9"
server_key_version: "encryptionkey@0"
I20260809 18:55:13.572379 18348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:13.587363 18348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:13.589200 18348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:13.589332 18348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:13.589532 18348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:13.589918 18441 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 18:55:13.591770 18348 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 18:55:13.591828 18348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 18:55:13.591862 18348 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 18:55:13.592458 18348 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 18:55:13.592490 18348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 18:55:13.592581 18441 tablet_bootstrap.cc:492] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Bootstrap starting.
I20260809 18:55:13.599164 18348 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:34423
I20260809 18:55:13.599304 18626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:34423 every 8 connection(s)
I20260809 18:55:13.599545 18348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:55:13.604182 18441 log.cc:824] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:13.605757 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 18348
I20260809 18:55:13.606019 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:34167
--local_ip_for_outbound_sockets=127.23.218.194
--tserver_master_addrs=127.23.218.254:38087
--webserver_port=45263
--webserver_interface=127.23.218.194
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:55:13.606712 18441 tablet_bootstrap.cc:492] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 18:55:13.607010 18441 tablet_bootstrap.cc:492] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Bootstrap complete.
I20260809 18:55:13.608214 18441 ts_tablet_manager.cc:1397] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260809 18:55:13.609606 18441 raft_consensus.cc:348] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:55:13.609929 18441 raft_consensus.cc:729] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7264dd200dcd4911a62df6df6f54f136, State: Initialized, Role: FOLLOWER
I20260809 18:55:13.610095 18441 consensus_queue.cc:260] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:55:13.610366 18441 ts_tablet_manager.cc:1428] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260809 18:55:13.665318 18629 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:55:13.600113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:34697)
0809 18:55:13.600275 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:13.600315 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:13.600717 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:13.665122 (+ 64405us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
W20260809 18:55:13.665557 18627 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:38087 (0 consecutive failures): Network error: Failed to ping master at 127.23.218.254:38087: Client connection negotiation failed: client connection to 127.23.218.254:38087: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 18:55:13.665861 18629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:13.665756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:40639)
0809 18:55:13.665803 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:13.665840 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 18:55:13.666163 18629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:13.666088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:34881)
0809 18:55:13.666123 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:13.666146 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 18:55:13.666308 18627 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 18:55:13.687232 18632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:13.687393 18632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:13.687414 18632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:13.688454 18632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:13.688485 18632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:13.688510 18632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:55:13.689929 18632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:34167
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=45263
--tserver_master_addrs=127.23.218.254:38087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:13.690097 18632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:13.690263 18632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:13.694720 18640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:13.694727 18638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:13.694715 18642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:13.694916 18632 server_base.cc:1034] running on GCE node
I20260809 18:55:13.695091 18632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:13.695292 18632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:13.696426 18632 hybrid_clock.cc:648] HybridClock initialized: now 1786301713696411 us; error 29 us; skew 500 ppm
Aug 09 18:55:13 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:13.698691 18632 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:55:13.699038 18632 webserver.cc:460] Webserver started at http://127.23.218.194:45263/ using document root <none> and password file <none>
I20260809 18:55:13.699216 18632 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:13.699254 18632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:13.700376 18632 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:55:13.700861 18659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.700994 18632 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.701079 18632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "931165f44627438b9bc6ebb7691ce2dd"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "c99ac5d2bb1460c6c22e4ff6e6f89e7e"
server_key_iv: "9b7389e0a235c1d6dd9151dc3408dc6e"
server_key_version: "encryptionkey@0"
I20260809 18:55:13.701362 18632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:13.719956 18632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:13.721508 18632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:13.721637 18632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:13.721822 18632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:13.722105 18632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:13.722137 18632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.722164 18632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:13.722186 18632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.727679 18632 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:34167
I20260809 18:55:13.727720 18844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:34167 every 8 connection(s)
I20260809 18:55:13.728012 18632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:55:13.728884 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:13.728566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:47917)
0809 18:55:13.728701 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:13.728760 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
W20260809 18:55:13.729198 18846 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:38087 (0 consecutive failures): Network error: Failed to ping master at 127.23.218.254:38087: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
I20260809 18:55:13.729468 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:13.729390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:49661)
0809 18:55:13.729430 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:13.729453 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 18:55:13.729664 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:13.729618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:47975)
0809 18:55:13.729637 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:13.729653 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 18:55:13.729739 18846 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 18:55:13.732230 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 18632
I20260809 18:55:13.732455 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:40459
--local_ip_for_outbound_sockets=127.23.218.195
--tserver_master_addrs=127.23.218.254:38087
--webserver_port=43867
--webserver_interface=127.23.218.195
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:45761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:13.811455 18850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:13.811635 18850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:13.811674 18850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:13.812757 18850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:13.812793 18850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:13.812827 18850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:55:13.814224 18850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:45761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:40459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=43867
--tserver_master_addrs=127.23.218.254:38087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:13.814467 18850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:13.814657 18850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:13.819098 18857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:13.819109 18855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:13.819103 18860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:13.819300 18850 server_base.cc:1034] running on GCE node
I20260809 18:55:13.819490 18850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:13.819700 18850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:13.820822 18850 hybrid_clock.cc:648] HybridClock initialized: now 1786301713820809 us; error 31 us; skew 500 ppm
Aug 09 18:55:13 dist-test-slave-6zbq krb5kdc[16443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:13.823088 18850 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:55:13.823472 18850 webserver.cc:460] Webserver started at http://127.23.218.195:43867/ using document root <none> and password file <none>
I20260809 18:55:13.823647 18850 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:13.823688 18850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:13.824855 18850 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:55:13.825328 18874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.825453 18850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.825532 18850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a"
format_stamp: "Formatted at 2026-08-09 18:54:32 on dist-test-slave-6zbq"
server_key: "11af8815e8c800ff2a8bedab56aa38f3"
server_key_iv: "50f927fa48cc69c9fb5bc27a92c205fe"
server_key_version: "encryptionkey@0"
I20260809 18:55:13.825807 18850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:13.849529 18850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:13.851204 18850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:13.851327 18850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:13.851518 18850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:13.851779 18850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:13.851811 18850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.851837 18850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:13.851855 18850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:13.857275 18850 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:40459
I20260809 18:55:13.857336 19064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:40459 every 8 connection(s)
I20260809 18:55:13.857620 18850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:55:13.858582 19068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:13.858232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:47235)
0809 18:55:13.858385 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:13.858453 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
W20260809 18:55:13.858942 19065 heartbeater.cc:644] Failed to heartbeat to 127.23.218.254:38087 (0 consecutive failures): Network error: Failed to ping master at 127.23.218.254:38087: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
I20260809 18:55:13.859159 19068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:13.859085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:46191)
0809 18:55:13.859120 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:13.859144 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 18:55:13.859356 19068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:13.859300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:57359)
0809 18:55:13.859328 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:13.859345 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 18:55:13.859434 19065 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 18:55:13.867384 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 18850
I20260809 18:55:14.666796 19071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:14.666595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:46497)
0809 18:55:14.666699 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:14.666729 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260809 18:55:14.730154 19075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:14.729985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:49641)
0809 18:55:14.730082 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:14.730109 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":55,"threads_started":1}
I20260809 18:55:14.859870 19077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:14.859681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:43591)
0809 18:55:14.859788 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:14.859821 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260809 18:55:15.331919 19078 raft_consensus.cc:482] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:15.332062 19078 raft_consensus.cc:504] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:55:15.332392 19078 leader_election.cc:288] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459), 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
Aug 09 18:55:15 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.195@KRBTEST.COM
Aug 09 18:55:15 dist-test-slave-6zbq krb5kdc[16443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.194@KRBTEST.COM
I20260809 18:55:15.346537 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:15.332648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:59003 (local address 127.23.218.194:34167)
0809 18:55:15.332844 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:15.332852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:15.332870 (+ 18us) server_negotiation.cc:407] Connection header received
0809 18:55:15.333315 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:15.333317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:15.333359 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:15.333469 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:15.334210 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:15.335180 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:15.336137 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:15.336519 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:15.336612 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:15.343531 (+ 6919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:15.343534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:15.343535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:15.343545 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:15.345134 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:15.345472 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:15.345475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:15.345476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:15.345557 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:15.345733 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:15.345734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:15.345734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:15.345948 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:15.346017 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:15.346286 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 18:55:15.346430 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260809 18:55:15.346539 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:15.332664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:33729 (local address 127.23.218.195:40459)
0809 18:55:15.332830 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:15.332842 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:15.332877 (+ 35us) server_negotiation.cc:407] Connection header received
0809 18:55:15.333318 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:15.333321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:15.333369 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:15.333483 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:15.334188 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:15.335188 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:15.336127 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:15.336515 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:15.336606 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:15.343147 (+ 6541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:15.343151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:15.343152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:15.343164 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:15.344941 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:15.345369 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:15.345408 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:15.345409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:15.345503 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:15.345727 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:15.345728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:15.345728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:15.345949 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:15.346017 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:15.346289 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 18:55:15.346429 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260809 18:55:15.346555 19080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:15.332552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:40459 (local address 127.23.218.193:33729)
0809 18:55:15.332776 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:15.332797 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:15.332834 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:15.333570 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:15.333571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:15.333889 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:15.334079 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:15.334085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:15.335294 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:15.335295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:15.336049 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:15.336051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:15.336595 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:15.336596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:15.336707 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:15.341648 (+ 4941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:15.341653 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:15.343045 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:15.345016 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:15.345023 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:15.345025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:15.345297 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:15.345569 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:15.345570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:15.345571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:15.345669 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:15.346013 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:15.346019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:15.346135 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 18:55:15.346219 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 18:55:15.346428 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":53,"thread_start_us":44,"threads_started":1}
I20260809 18:55:15.347164 19081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:15.332526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34167 (local address 127.23.218.193:59003)
0809 18:55:15.332662 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:15.332671 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:15.332717 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:15.333536 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:15.333538 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:15.333888 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:15.334079 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:15.334085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:15.335274 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:15.335275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:15.336050 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:15.336052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:15.336601 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:15.336602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:15.336704 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:15.341644 (+ 4940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:15.341652 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:15.343428 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:15.345205 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:15.345206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:15.345207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:15.345402 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:15.345617 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:15.345618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:15.345618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:15.345684 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:15.346012 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:15.346019 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:15.346135 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 18:55:15.346220 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 18:55:15.346500 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":176,"thread_start_us":67,"threads_started":1}
W20260809 18:55:15.347254 18437 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
W20260809 18:55:15.347744 18435 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
I20260809 18:55:15.347791 18435 leader_election.cc:302] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7264dd200dcd4911a62df6df6f54f136; no voters: 59fb4a5b22a34a94ae7ce9fb171aac9a, 931165f44627438b9bc6ebb7691ce2dd
I20260809 18:55:15.347879 19078 raft_consensus.cc:2726] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:55:15.667285 19081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:15.667164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:54809)
0809 18:55:15.667217 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:15.667242 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 18:55:15.730758 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:15.730570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:59393)
0809 18:55:15.730675 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:15.730717 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260809 18:55:15.860409 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:15.860225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:40505)
0809 18:55:15.860343 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:15.860375 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260809 18:55:16.667906 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:16.667709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:32771)
0809 18:55:16.667827 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:16.667861 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260809 18:55:16.731290 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:16.731085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:38259)
0809 18:55:16.731212 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:16.731244 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260809 18:55:16.860963 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:16.860775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:41421)
0809 18:55:16.860894 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:16.860927 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260809 18:55:17.101912 19095 raft_consensus.cc:482] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:17.102007 19095 raft_consensus.cc:504] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:55:17.102203 19095 leader_election.cc:288] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459), 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
W20260809 18:55:17.102592 18435 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
W20260809 18:55:17.102659 18437 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
I20260809 18:55:17.102726 18437 leader_election.cc:302] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7264dd200dcd4911a62df6df6f54f136; no voters: 59fb4a5b22a34a94ae7ce9fb171aac9a, 931165f44627438b9bc6ebb7691ce2dd
I20260809 18:55:17.102802 19095 raft_consensus.cc:2726] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:55:17.668503 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:17.668299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:58413)
0809 18:55:17.668429 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:17.668460 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260809 18:55:17.731808 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:17.731631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:47019)
0809 18:55:17.731741 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:17.731773 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260809 18:55:17.861522 19100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:17.861338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:57007)
0809 18:55:17.861460 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:17.861490 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260809 18:55:18.669171 19102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:18.668961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:48835)
0809 18:55:18.669080 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:18.669119 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260809 18:55:18.732324 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:18.732141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:58613)
0809 18:55:18.732256 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:18.732288 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260809 18:55:18.744800 19107 raft_consensus.cc:482] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:18.744885 19107 raft_consensus.cc:504] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:55:18.745045 19107 leader_election.cc:288] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459), 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
W20260809 18:55:18.745381 18435 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
W20260809 18:55:18.745462 18437 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
I20260809 18:55:18.745502 18437 leader_election.cc:302] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7264dd200dcd4911a62df6df6f54f136; no voters: 59fb4a5b22a34a94ae7ce9fb171aac9a, 931165f44627438b9bc6ebb7691ce2dd
I20260809 18:55:18.745564 19107 raft_consensus.cc:2726] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:55:18.862057 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:18.861883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:32797)
0809 18:55:18.861991 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:18.862022 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260809 18:55:19.669853 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:19.669606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:36365)
0809 18:55:19.669768 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:19.669805 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260809 18:55:19.732898 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:19.732720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:49611)
0809 18:55:19.732828 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:19.732862 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260809 18:55:19.862635 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:19.862425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:40193)
0809 18:55:19.862562 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:19.862599 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
I20260809 18:55:20.670593 19115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:20.670363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:38071)
0809 18:55:20.670515 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:20.670545 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260809 18:55:20.733462 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:20.733281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:52225)
0809 18:55:20.733399 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:20.733427 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260809 18:55:20.863216 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:20.863035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:35123)
0809 18:55:20.863148 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:20.863178 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260809 18:55:21.671226 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:21.671017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:60665)
0809 18:55:21.671146 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:21.671178 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260809 18:55:21.734035 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:21.733841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:43835)
0809 18:55:21.733970 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:21.733998 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260809 18:55:21.863782 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:21.863603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:38717)
0809 18:55:21.863715 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:21.863746 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260809 18:55:21.865592 19123 raft_consensus.cc:482] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:21.865670 19123 raft_consensus.cc:504] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:55:21.865833 19123 leader_election.cc:288] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459), 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
W20260809 18:55:21.866140 18437 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
W20260809 18:55:21.866220 18435 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
I20260809 18:55:21.866261 18435 leader_election.cc:302] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7264dd200dcd4911a62df6df6f54f136; no voters: 59fb4a5b22a34a94ae7ce9fb171aac9a, 931165f44627438b9bc6ebb7691ce2dd
I20260809 18:55:21.866315 19123 raft_consensus.cc:2726] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:55:22.671887 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:22.671686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:56075)
0809 18:55:22.671806 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:22.671840 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260809 18:55:22.734601 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:22.734423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:55171)
0809 18:55:22.734528 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:22.734564 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260809 18:55:22.864307 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:22.864127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:42303)
0809 18:55:22.864240 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:22.864272 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
I20260809 18:55:23.672488 19128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:23.672275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:50295)
0809 18:55:23.672404 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:23.672439 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260809 18:55:23.735234 19129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:23.735045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:57907)
0809 18:55:23.735165 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:23.735198 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260809 18:55:23.830235 19130 raft_consensus.cc:482] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:23.830313 19130 raft_consensus.cc:504] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:55:23.830513 19130 leader_election.cc:288] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459), 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
W20260809 18:55:23.830770 18437 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
W20260809 18:55:23.830847 18435 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
I20260809 18:55:23.830904 18435 leader_election.cc:302] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7264dd200dcd4911a62df6df6f54f136; no voters: 59fb4a5b22a34a94ae7ce9fb171aac9a, 931165f44627438b9bc6ebb7691ce2dd
I20260809 18:55:23.830979 19130 raft_consensus.cc:2726] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:55:23.864847 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:23.864643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:55899)
0809 18:55:23.864770 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:23.864800 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260809 18:55:24.673033 19132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:24.672839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:56775)
0809 18:55:24.672957 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:24.672986 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260809 18:55:24.735772 19133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:24.735593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:46495)
0809 18:55:24.735702 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:24.735735 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260809 18:55:24.865381 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:24.865195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:49685)
0809 18:55:24.865315 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:24.865344 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260809 18:55:25.673592 19137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:25.673377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:60565)
0809 18:55:25.673509 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:25.673545 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":58,"threads_started":1}
I20260809 18:55:25.736311 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:25.736128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:54031)
0809 18:55:25.736243 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:25.736273 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260809 18:55:25.865926 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:25.865718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:47171)
0809 18:55:25.865852 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:25.865886 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260809 18:55:25.935599 19146 raft_consensus.cc:482] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:25.935695 19146 raft_consensus.cc:504] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59fb4a5b22a34a94ae7ce9fb171aac9a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 40459 } } peers { permanent_uuid: "931165f44627438b9bc6ebb7691ce2dd" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34167 } } peers { permanent_uuid: "7264dd200dcd4911a62df6df6f54f136" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 34423 } }
I20260809 18:55:25.935864 19146 leader_election.cc:288] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459), 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167)
W20260809 18:55:25.936225 18435 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 931165f44627438b9bc6ebb7691ce2dd (127.23.218.194:34167): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
W20260809 18:55:25.936307 18437 leader_election.cc:341] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 59fb4a5b22a34a94ae7ce9fb171aac9a (127.23.218.195:40459): Not found: Tablet not found: e7f03ddf09ef4bdd8ac446aff953310a
I20260809 18:55:25.936347 18437 leader_election.cc:302] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7264dd200dcd4911a62df6df6f54f136; no voters: 59fb4a5b22a34a94ae7ce9fb171aac9a, 931165f44627438b9bc6ebb7691ce2dd
I20260809 18:55:25.936406 19146 raft_consensus.cc:2726] T e7f03ddf09ef4bdd8ac446aff953310a P 7264dd200dcd4911a62df6df6f54f136 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 18:55:26.674181 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:26.673973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:44121)
0809 18:55:26.674093 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:26.674123 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260809 18:55:26.736904 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:26.736691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:50359)
0809 18:55:26.736817 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:26.736851 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260809 18:55:26.866583 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:26.866378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:41825)
0809 18:55:26.866508 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:26.866543 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260809 18:55:27.674823 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:27.674604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:34695)
0809 18:55:27.674741 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:27.674776 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260809 18:55:27.737442 19154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:27.737263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:55083)
0809 18:55:27.737374 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:27.737406 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260809 18:55:27.867154 19155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:27.866945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:49129)
0809 18:55:27.867077 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:27.867112 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260809 18:55:28.675426 19156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:28.675240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.193:42695)
0809 18:55:28.675352 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:28.675381 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
I20260809 18:55:28.738005 19157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:28.737837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.194:35047)
0809 18:55:28.737938 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:28.737971 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
I20260809 18:55:28.867750 19159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:28.867572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:38087 (local address 127.23.218.195:55497)
0809 18:55:28.867688 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:28.867715 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.254:38087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260809 18:55:28.877316 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 18348
I20260809 18:55:28.885169 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 18632
I20260809 18:55:28.900321 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 18850
2026-08-09T18:55:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58300 ms)
[----------] 30 tests from SecurityITest (229368 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:28 dist-test-slave-6zbq krb5kdc[19165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:28 dist-test-slave-6zbq krb5kdc[19165](info): set up 2 sockets
Aug 09 18:55:28 dist-test-slave-6zbq krb5kdc[19165](info): commencing operation
krb5kdc: starting...
W20260809 18:55:30.048532 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:30 dist-test-slave-6zbq krb5kdc[19165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:55:30.154476 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:43615
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:43615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:30.232306 19183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:30.232481 19183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:30.232511 19183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:30.233583 19183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:30.233618 19183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:30.233637 19183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:30.233652 19183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:30.233668 19183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:30.235110 19183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:43615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:43615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:30.235328 19183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:30.235510 19183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:30.239907 19189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:30.239986 19188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:30.239914 19192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:30.240082 19183 server_base.cc:1034] running on GCE node
I20260809 18:55:30.240388 19183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:30.240622 19183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:30.241755 19183 hybrid_clock.cc:648] HybridClock initialized: now 1786301730241743 us; error 28 us; skew 500 ppm
Aug 09 18:55:30 dist-test-slave-6zbq krb5kdc[19165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301730, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:30.243935 19183 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:55:30.244330 19183 webserver.cc:460] Webserver started at http://127.23.218.254:45413/ using document root <none> and password file <none>
I20260809 18:55:30.244510 19183 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:30.244555 19183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:30.244683 19183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:30.245594 19183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "e62f7eb4e9a9473883a0298bc3e16da9"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "209fa470a1cb18d60bcacba8268c0960"
server_key_iv: "4220834412e11b7da30d8ce4c338a4f4"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.245916 19183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "e62f7eb4e9a9473883a0298bc3e16da9"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "209fa470a1cb18d60bcacba8268c0960"
server_key_iv: "4220834412e11b7da30d8ce4c338a4f4"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.247201 19183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:55:30.247812 19206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.247967 19183 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:55:30.248039 19183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "e62f7eb4e9a9473883a0298bc3e16da9"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "209fa470a1cb18d60bcacba8268c0960"
server_key_iv: "4220834412e11b7da30d8ce4c338a4f4"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.248106 19183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:30.264227 19183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:30.265758 19183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:30.265872 19183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:30.269498 19183 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:43615
I20260809 18:55:30.269537 19292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:43615 every 8 connection(s)
I20260809 18:55:30.269821 19183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:30.270599 19293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:30.272594 19293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9: Bootstrap starting.
I20260809 18:55:30.273095 19293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:30.273356 19293 log.cc:824] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:30.273945 19293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9: No bootstrap required, opened a new log
I20260809 18:55:30.275501 19293 raft_consensus.cc:348] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62f7eb4e9a9473883a0298bc3e16da9" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43615 } }
I20260809 18:55:30.275624 19293 raft_consensus.cc:374] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:30.275656 19293 raft_consensus.cc:729] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e62f7eb4e9a9473883a0298bc3e16da9, State: Initialized, Role: FOLLOWER
I20260809 18:55:30.275790 19293 consensus_queue.cc:260] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62f7eb4e9a9473883a0298bc3e16da9" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43615 } }
I20260809 18:55:30.275863 19293 raft_consensus.cc:388] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:30.275903 19293 raft_consensus.cc:482] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:30.275949 19293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:30.276455 19293 raft_consensus.cc:504] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62f7eb4e9a9473883a0298bc3e16da9" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43615 } }
I20260809 18:55:30.276543 19293 leader_election.cc:302] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e62f7eb4e9a9473883a0298bc3e16da9; no voters:
I20260809 18:55:30.276690 19293 leader_election.cc:288] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:30.276722 19296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:30.276860 19296 raft_consensus.cc:686] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [term 1 LEADER]: Becoming Leader. State: Replica: e62f7eb4e9a9473883a0298bc3e16da9, State: Running, Role: LEADER
I20260809 18:55:30.276924 19293 sys_catalog.cc:564] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:30.276974 19296 consensus_queue.cc:237] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62f7eb4e9a9473883a0298bc3e16da9" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43615 } }
I20260809 18:55:30.277329 19298 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e62f7eb4e9a9473883a0298bc3e16da9. Latest consensus state: current_term: 1 leader_uuid: "e62f7eb4e9a9473883a0298bc3e16da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62f7eb4e9a9473883a0298bc3e16da9" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43615 } } }
I20260809 18:55:30.277434 19297 sys_catalog.cc:455] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e62f7eb4e9a9473883a0298bc3e16da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e62f7eb4e9a9473883a0298bc3e16da9" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 43615 } } }
I20260809 18:55:30.277479 19298 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:30.277518 19297 sys_catalog.cc:458] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:30.278020 19308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:30.278281 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 19183
I20260809 18:55:30.278419 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:30.278566 19308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:30.278620 24427 external_mini_cluster.cc:1428] Setting key 0ab58e5a8be132fc21e0e1820ca6234a
I20260809 18:55:30.280733 19308 catalog_manager.cc:1347] Generated new cluster ID: c6ed3da1dc894285bf45ec573d3b773d
I20260809 18:55:30.280782 19308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:55:30 dist-test-slave-6zbq krb5kdc[19165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:30.289301 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.279178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.23.218.254:43615)
0809 18:55:30.279326 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.279335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.279363 (+ 28us) server_negotiation.cc:407] Connection header received
0809 18:55:30.279468 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.279473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.279533 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.279669 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:30.280363 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.281619 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.282688 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.283273 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.283362 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.286026 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:30.286030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:30.286030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:30.286043 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:30.287808 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.288177 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.288179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.288180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.288286 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.288464 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.288464 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.288465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.288730 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:30.288864 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.288969 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.289092 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260809 18:55:30.290735 19308 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:30.291198 19308 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:30.308808 19308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9: Generated new TSK 0
I20260809 18:55:30.308946 19308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:55:30.332679 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43615
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:30.410923 19324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:30.411108 19324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:30.411130 19324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:30.412174 19324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:30.412206 19324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:30.412230 19324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:55:30.413669 19324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:30.413838 19324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:30.413997 19324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:30.418617 19339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:30.418608 19345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:30.418725 19338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:30.418627 19324 server_base.cc:1034] running on GCE node
I20260809 18:55:30.418977 19324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:30.419201 19324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:30.420336 19324 hybrid_clock.cc:648] HybridClock initialized: now 1786301730420323 us; error 27 us; skew 500 ppm
Aug 09 18:55:30 dist-test-slave-6zbq krb5kdc[19165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301730, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:30.422663 19324 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:55:30.423041 19324 webserver.cc:460] Webserver started at http://127.23.218.193:34399/ using document root <none> and password file <none>
I20260809 18:55:30.423233 19324 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:30.423280 19324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:30.423395 19324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:30.424330 19324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "e089a9245d1b4aeebfe42c882bbe9a73"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "8c224bac295dcf0860a2c85dc9e7bd7c"
server_key_iv: "b0e8eebc8557118c3d4b35332fd0f9ef"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.424654 19324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "e089a9245d1b4aeebfe42c882bbe9a73"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "8c224bac295dcf0860a2c85dc9e7bd7c"
server_key_iv: "b0e8eebc8557118c3d4b35332fd0f9ef"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.425824 19324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 18:55:30.426501 19354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.426651 19324 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.426731 19324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "e089a9245d1b4aeebfe42c882bbe9a73"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "8c224bac295dcf0860a2c85dc9e7bd7c"
server_key_iv: "b0e8eebc8557118c3d4b35332fd0f9ef"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.426796 19324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:30.444392 19324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:30.445955 19324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:30.446079 19324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:30.446280 19324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:30.446563 19324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:30.446597 19324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.446622 19324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:30.446647 19324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.452030 19324 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:40663
I20260809 18:55:30.452095 19543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:40663 every 8 connection(s)
I20260809 18:55:30.452346 19324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:55:30.456741 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 19324
I20260809 18:55:30.456845 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:55:30.457011 24427 external_mini_cluster.cc:1428] Setting key a60861860377e5224a88e277e3cd9756
Aug 09 18:55:30 dist-test-slave-6zbq krb5kdc[19165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301730, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:30.461225 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.452931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:53751 (local address 127.23.218.254:43615)
0809 18:55:30.452970 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.452972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.453169 (+ 197us) server_negotiation.cc:407] Connection header received
0809 18:55:30.453631 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.453632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.453648 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.453712 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:30.454634 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.455121 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.456049 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.456254 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.456308 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.458564 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:30.458566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:30.458566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:30.458573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:30.459985 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.460355 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.460356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.460356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.460409 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.460616 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.460618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.460618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.460732 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:30.460792 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.461033 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.461140 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:55:30.461364 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.452908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:43615 (local address 127.23.218.193:53751)
0809 18:55:30.453065 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:30.453096 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:30.453541 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:30.453790 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:30.453794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:30.454327 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:30.454549 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.454553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.455206 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.455208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.455969 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.455971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.456323 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.456324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.456432 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.456864 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:30.456874 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:30.458483 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:30.460069 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:30.460076 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:30.460078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:30.460290 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:30.460461 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:30.460461 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:30.460461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:30.460545 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:30.460793 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:30.460799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:30.460917 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 18:55:30.461002 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 18:55:30.461169 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260809 18:55:30.461822 19544 heartbeater.cc:347] Connected to a master server at 127.23.218.254:43615
I20260809 18:55:30.461916 19544 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:30.462073 19544 heartbeater.cc:511] Master 127.23.218.254:43615 requested a full tablet report, sending...
I20260809 18:55:30.462563 19235 ts_manager.cc:194] Registered new tserver with Master: e089a9245d1b4aeebfe42c882bbe9a73 (127.23.218.193:40663)
I20260809 18:55:30.463143 19235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:53751
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:55:30.479082 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43615
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:30.557889 19550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:30.558072 19550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:30.558110 19550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:30.559370 19550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:30.559409 19550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:30.559444 19550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:55:30.560863 19550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:30.561106 19550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:30.561312 19550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:30.565933 19561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:30.566039 19550 server_base.cc:1034] running on GCE node
W20260809 18:55:30.565948 19555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:30.565932 19556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:30.566319 19550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:30.566562 19550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:30.567698 19550 hybrid_clock.cc:648] HybridClock initialized: now 1786301730567684 us; error 29 us; skew 500 ppm
Aug 09 18:55:30 dist-test-slave-6zbq krb5kdc[19165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301730, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:30.569916 19550 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:55:30.570288 19550 webserver.cc:460] Webserver started at http://127.23.218.194:40611/ using document root <none> and password file <none>
I20260809 18:55:30.570520 19550 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:30.570564 19550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:30.570679 19550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:30.571609 19550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "c370cafdc27f478095f0a9430220f390"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "65026547449f520563d4e91c4814f8dd"
server_key_iv: "41b4c23da79fa8c8f6dd8e58972d2556"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.571907 19550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "c370cafdc27f478095f0a9430220f390"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "65026547449f520563d4e91c4814f8dd"
server_key_iv: "41b4c23da79fa8c8f6dd8e58972d2556"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.573096 19550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:30.573699 19572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.573843 19550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:30.573923 19550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "c370cafdc27f478095f0a9430220f390"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "65026547449f520563d4e91c4814f8dd"
server_key_iv: "41b4c23da79fa8c8f6dd8e58972d2556"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.573982 19550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:30.624629 19550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:30.626655 19550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:30.626796 19550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:30.626988 19550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:30.627270 19550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:30.627303 19550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.627331 19550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:30.627353 19550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.632686 19550 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:35979
I20260809 18:55:30.632750 19781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:35979 every 8 connection(s)
I20260809 18:55:30.632993 19550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:55:30.633790 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 19550
I20260809 18:55:30.633898 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:55:30.634063 24427 external_mini_cluster.cc:1428] Setting key 4f284f6d6eb5782f49fec336623ed2f7
Aug 09 18:55:30 dist-test-slave-6zbq krb5kdc[19165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301730, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:30.641333 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.633610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:42413 (local address 127.23.218.254:43615)
0809 18:55:30.633657 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.633658 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.633833 (+ 175us) server_negotiation.cc:407] Connection header received
0809 18:55:30.634308 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.634309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.634322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.634399 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:30.635263 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.635751 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.636687 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.636928 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.636982 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.638954 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:30.638955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:30.638956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:30.638963 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:30.640130 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.640470 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.640471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.640472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.640529 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.640743 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.640744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.640744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.640853 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:30.640907 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.641163 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.641247 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:30.641465 19787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.633568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:43615 (local address 127.23.218.194:42413)
0809 18:55:30.633729 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:30.633762 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:30.634214 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:30.634467 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:30.634471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:30.634991 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:30.635171 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.635175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.635832 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.635833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.636603 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.636606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.637011 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.637012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.637134 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.637507 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:30.637514 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:30.638866 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:30.640199 (+ 1333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:30.640206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:30.640208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:30.640404 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:30.640580 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:30.640581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:30.640581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:30.640687 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:30.640906 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:30.640912 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:30.641036 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 18:55:30.641132 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 18:55:30.641274 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260809 18:55:30.642010 19782 heartbeater.cc:347] Connected to a master server at 127.23.218.254:43615
I20260809 18:55:30.642093 19782 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
I20260809 18:55:30.642258 19782 heartbeater.cc:511] Master 127.23.218.254:43615 requested a full tablet report, sending...
I20260809 18:55:30.642635 19235 ts_manager.cc:194] Registered new tserver with Master: c370cafdc27f478095f0a9430220f390 (127.23.218.194:35979)
I20260809 18:55:30.642997 19235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:42413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:55:30.656654 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43615
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:30.734879 19793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:30.735044 19793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:30.735066 19793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:30.736153 19793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:30.736191 19793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:30.736217 19793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:55:30.737596 19793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:43615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:30.737787 19793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:30.737977 19793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:30.742429 19798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:30.742451 19804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:30.742456 19802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:30.742554 19793 server_base.cc:1034] running on GCE node
I20260809 18:55:30.742815 19793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:30.743032 19793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:30.744167 19793 hybrid_clock.cc:648] HybridClock initialized: now 1786301730744154 us; error 28 us; skew 500 ppm
Aug 09 18:55:30 dist-test-slave-6zbq krb5kdc[19165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301730, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:30.746407 19793 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:55:30.746769 19793 webserver.cc:460] Webserver started at http://127.23.218.195:40129/ using document root <none> and password file <none>
I20260809 18:55:30.746942 19793 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:30.746987 19793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:30.747112 19793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:30.748054 19793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "2c40e660d4724becbd468b2a095b604a"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "847f41e346287cc644cb6a64c938a4fb"
server_key_iv: "138ddd202c0a70a768f702c4e59b133a"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.748378 19793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "2c40e660d4724becbd468b2a095b604a"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "847f41e346287cc644cb6a64c938a4fb"
server_key_iv: "138ddd202c0a70a768f702c4e59b133a"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.749527 19793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:30.750147 19813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.750295 19793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.750394 19793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "2c40e660d4724becbd468b2a095b604a"
format_stamp: "Formatted at 2026-08-09 18:55:30 on dist-test-slave-6zbq"
server_key: "847f41e346287cc644cb6a64c938a4fb"
server_key_iv: "138ddd202c0a70a768f702c4e59b133a"
server_key_version: "encryptionkey@0"
I20260809 18:55:30.750461 19793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:30.762279 19793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:30.764217 19793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:30.764340 19793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:30.764514 19793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:30.764765 19793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:30.764797 19793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.764823 19793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:30.764846 19793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:30.770112 19793 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:37913
I20260809 18:55:30.770167 20015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:37913 every 8 connection(s)
I20260809 18:55:30.770460 19793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
Aug 09 18:55:30 dist-test-slave-6zbq krb5kdc[19165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301730, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:30.778311 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.771054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:57997 (local address 127.23.218.254:43615)
0809 18:55:30.771100 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.771102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.771271 (+ 169us) server_negotiation.cc:407] Connection header received
0809 18:55:30.771693 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.771694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.771704 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.771738 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:30.772524 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.772995 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.773875 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.774044 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.774085 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.775875 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:30.775876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:30.775876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:30.775883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:30.777062 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.777388 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.777389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.777389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.777434 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.777718 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.777719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.777719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.777830 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:30.777889 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.778142 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.778223 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:55:30.778494 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.771030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:43615 (local address 127.23.218.195:57997)
0809 18:55:30.771170 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:30.771201 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:30.771606 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:30.771790 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:30.771793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:30.772291 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:30.772458 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.772463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.773048 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.773049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.773808 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.773811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.774109 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.774110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.774204 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.774622 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:30.774630 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:30.775796 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:30.777134 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:30.777141 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:30.777142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:30.777333 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:30.777510 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:30.777511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:30.777511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:30.777657 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:30.777885 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:30.777893 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:30.778019 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 18:55:30.778105 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 18:55:30.778256 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":82,"threads_started":1}
I20260809 18:55:30.778906 20016 heartbeater.cc:347] Connected to a master server at 127.23.218.254:43615
I20260809 18:55:30.778990 20016 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:30.779146 20016 heartbeater.cc:511] Master 127.23.218.254:43615 requested a full tablet report, sending...
I20260809 18:55:30.779464 19235 ts_manager.cc:194] Registered new tserver with Master: 2c40e660d4724becbd468b2a095b604a (127.23.218.195:37913)
I20260809 18:55:30.779804 19235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:57997
I20260809 18:55:30.780517 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 19793
I20260809 18:55:30.780616 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:55:30.780769 24427 external_mini_cluster.cc:1428] Setting key ae556bc96c0256ec6ee1404ee3128ed1
I20260809 18:55:30.781479 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:55:30.787637 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.782679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.23.218.254:43615)
0809 18:55:30.782722 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.782723 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.782833 (+ 110us) server_negotiation.cc:407] Connection header received
0809 18:55:30.782891 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.782892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.782902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.782941 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:30.783456 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.783893 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.784570 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.784749 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.784798 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.785754 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:30.785755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:30.785755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:30.785761 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:30.786700 (+ 939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.786978 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.786979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.786979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.787025 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:30.787184 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:30.787185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:30.787185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:30.787305 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:30.787389 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.787492 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.787558 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:55:30.788882 19235 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:55:30.790361 19235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:55:30.796193 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.792330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.23.218.193:40663)
0809 18:55:30.792520 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.792524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.792619 (+ 95us) server_negotiation.cc:407] Connection header received
0809 18:55:30.792725 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.792728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.792744 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.792814 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:30.793047 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.793768 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.795017 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.795722 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.795768 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.795816 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.796063 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.796103 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":88,"threads_started":1}
I20260809 18:55:30.796200 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.792376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35848 (local address 127.23.218.195:37913)
0809 18:55:30.792595 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.792599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.792802 (+ 203us) server_negotiation.cc:407] Connection header received
0809 18:55:30.792852 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.792853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.792867 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.792930 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:30.793092 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.794149 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.795415 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.795864 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.795931 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.795982 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.796090 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.796127 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":68,"threads_started":1}
I20260809 18:55:30.796208 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.792164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:37913 (local address 127.0.0.1:35848)
0809 18:55:30.792731 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:30.792739 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:30.792787 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:30.792993 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:30.792993 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:30.792995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:30.793025 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.793026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.794259 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.794260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.795331 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.795334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.795953 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.795954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.796019 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.796023 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:30.796071 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 18:55:30.796147 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":2,"thread_start_us":49,"threads_started":1}
I20260809 18:55:30.796378 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.792161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40663 (local address 127.0.0.1:43152)
0809 18:55:30.792523 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:30.792560 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:30.792632 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:30.792901 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:30.792905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:30.792906 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:30.792965 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.792967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.793857 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.793858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.794926 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.794929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.795872 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.795873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.795959 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.795971 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 18:55:30.796271 (+ 300us) client_negotiation.cc:240] Negotiation successful
0809 18:55:30.796314 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":71,"threads_started":1}
I20260809 18:55:30.796998 19893 tablet_service.cc:1467] Processing CreateTablet for tablet be9cb7b561f14cb6808136664219149d (DEFAULT_TABLE table=test-table [id=7b0ff66c35424c64986fe3f9c3945fbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:30.797061 19440 tablet_service.cc:1467] Processing CreateTablet for tablet be9cb7b561f14cb6808136664219149d (DEFAULT_TABLE table=test-table [id=7b0ff66c35424c64986fe3f9c3945fbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:30.797313 19893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9cb7b561f14cb6808136664219149d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:30.797389 19440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9cb7b561f14cb6808136664219149d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:30.797533 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.792164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:35979 (local address 127.0.0.1:41394)
0809 18:55:30.792569 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:30.792580 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:30.792643 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:30.792917 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:30.792918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:30.792920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:30.792974 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.792977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.795413 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.795414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.796435 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.796439 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.797328 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.797329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.797388 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.797392 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:30.797439 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 18:55:30.797473 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":51,"threads_started":1}
I20260809 18:55:30.797760 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.792393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41394 (local address 127.23.218.194:35979)
0809 18:55:30.792560 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.792564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.792636 (+ 72us) server_negotiation.cc:407] Connection header received
0809 18:55:30.792713 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.792715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.792731 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.792812 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:30.793053 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.794165 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.796501 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.797169 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.797241 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.797291 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.797634 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.797677 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260809 18:55:30.798382 19659 tablet_service.cc:1467] Processing CreateTablet for tablet be9cb7b561f14cb6808136664219149d (DEFAULT_TABLE table=test-table [id=7b0ff66c35424c64986fe3f9c3945fbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:30.798691 19659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9cb7b561f14cb6808136664219149d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:30.800187 20040 tablet_bootstrap.cc:492] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: Bootstrap starting.
I20260809 18:55:30.800534 20041 tablet_bootstrap.cc:492] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73: Bootstrap starting.
I20260809 18:55:30.800861 20040 tablet_bootstrap.cc:654] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:30.801215 20041 tablet_bootstrap.cc:654] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:30.801431 20040 log.cc:824] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:30.801651 20042 tablet_bootstrap.cc:492] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: Bootstrap starting.
I20260809 18:55:30.801805 20041 log.cc:824] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:30.802258 20042 tablet_bootstrap.cc:654] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:30.802284 20040 tablet_bootstrap.cc:492] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: No bootstrap required, opened a new log
I20260809 18:55:30.802330 20040 ts_tablet_manager.cc:1397] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:55:30.802757 20042 log.cc:824] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:30.803382 20042 tablet_bootstrap.cc:492] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: No bootstrap required, opened a new log
I20260809 18:55:30.803452 20042 ts_tablet_manager.cc:1397] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 18:55:30.803995 20040 raft_consensus.cc:348] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } }
I20260809 18:55:30.804129 20040 raft_consensus.cc:374] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:30.804162 20040 raft_consensus.cc:729] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c40e660d4724becbd468b2a095b604a, State: Initialized, Role: FOLLOWER
I20260809 18:55:30.804283 20040 consensus_queue.cc:260] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } }
I20260809 18:55:30.804543 20040 ts_tablet_manager.cc:1428] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 18:55:30.804596 20016 heartbeater.cc:503] Master 127.23.218.254:43615 was elected leader, sending a full tablet report...
I20260809 18:55:30.804971 20042 raft_consensus.cc:348] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } }
I20260809 18:55:30.805109 20042 raft_consensus.cc:374] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:30.805114 20041 tablet_bootstrap.cc:492] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73: No bootstrap required, opened a new log
I20260809 18:55:30.805142 20042 raft_consensus.cc:729] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c370cafdc27f478095f0a9430220f390, State: Initialized, Role: FOLLOWER
I20260809 18:55:30.805171 20041 ts_tablet_manager.cc:1397] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 18:55:30.805266 20042 consensus_queue.cc:260] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } }
I20260809 18:55:30.805531 19782 heartbeater.cc:503] Master 127.23.218.254:43615 was elected leader, sending a full tablet report...
I20260809 18:55:30.805819 20042 ts_tablet_manager.cc:1428] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 18:55:30.806830 20041 raft_consensus.cc:348] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } }
I20260809 18:55:30.806955 20041 raft_consensus.cc:374] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:30.806975 20041 raft_consensus.cc:729] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e089a9245d1b4aeebfe42c882bbe9a73, State: Initialized, Role: FOLLOWER
I20260809 18:55:30.807052 20041 consensus_queue.cc:260] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } }
I20260809 18:55:30.807271 20041 ts_tablet_manager.cc:1428] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:55:30.807332 19544 heartbeater.cc:503] Master 127.23.218.254:43615 was elected leader, sending a full tablet report...
W20260809 18:55:30.883461 19783 tablet.cc:2369] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:55:30.934762 20046 raft_consensus.cc:482] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:30.934883 20046 raft_consensus.cc:504] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } }
I20260809 18:55:30.935230 20046 leader_election.cc:288] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e089a9245d1b4aeebfe42c882bbe9a73 (127.23.218.193:40663), c370cafdc27f478095f0a9430220f390 (127.23.218.194:35979)
I20260809 18:55:30.937778 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.935445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:48677 (local address 127.23.218.194:35979)
0809 18:55:30.935482 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.935484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.935491 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:55:30.935536 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.935537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.935546 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.935589 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:30.935719 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.936190 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.937102 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.937481 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.937543 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.937552 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.937683 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.937715 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:55:30.937827 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.935442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:34745 (local address 127.23.218.193:40663)
0809 18:55:30.935482 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.935484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.935612 (+ 128us) server_negotiation.cc:407] Connection header received
0809 18:55:30.935666 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.935667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.935675 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.935716 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:30.935854 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.936299 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.937189 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.937531 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.937591 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.937599 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.937726 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.937761 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:55:30.937839 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.935375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:35979 (local address 127.23.218.195:48677)
0809 18:55:30.935432 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:30.935442 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:30.935482 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:30.935597 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:30.935598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:30.935599 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:30.935667 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.935669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.936258 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.936260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.937019 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.937021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.937547 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.937548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.937623 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.937626 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:30.937672 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 18:55:30.937712 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":40}
I20260809 18:55:30.937855 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.935375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40663 (local address 127.23.218.195:34745)
0809 18:55:30.935541 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:30.935550 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:30.935607 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:30.935724 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:30.935724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:30.935726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:30.935797 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.935798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.936365 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.936366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.937105 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:30.937108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.937598 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.937599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:30.937664 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.937667 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:30.937711 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 18:55:30.937748 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":11,"thread_start_us":112,"threads_started":1}
I20260809 18:55:30.938148 19692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9cb7b561f14cb6808136664219149d" candidate_uuid: "2c40e660d4724becbd468b2a095b604a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c370cafdc27f478095f0a9430220f390" is_pre_election: true
I20260809 18:55:30.938192 19474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9cb7b561f14cb6808136664219149d" candidate_uuid: "2c40e660d4724becbd468b2a095b604a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" is_pre_election: true
I20260809 18:55:30.938273 19692 raft_consensus.cc:2454] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c40e660d4724becbd468b2a095b604a in term 0.
I20260809 18:55:30.938362 19474 raft_consensus.cc:2454] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c40e660d4724becbd468b2a095b604a in term 0.
I20260809 18:55:30.938529 19822 leader_election.cc:302] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c40e660d4724becbd468b2a095b604a, c370cafdc27f478095f0a9430220f390; no voters:
I20260809 18:55:30.938663 20046 raft_consensus.cc:2781] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:55:30.938722 20046 raft_consensus.cc:482] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:55:30.938740 20046 raft_consensus.cc:3037] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:30.939412 20046 raft_consensus.cc:504] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } }
I20260809 18:55:30.939525 20046 leader_election.cc:288] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [CANDIDATE]: Term 1 election: Requested vote from peers e089a9245d1b4aeebfe42c882bbe9a73 (127.23.218.193:40663), c370cafdc27f478095f0a9430220f390 (127.23.218.194:35979)
I20260809 18:55:30.939651 19692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9cb7b561f14cb6808136664219149d" candidate_uuid: "2c40e660d4724becbd468b2a095b604a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c370cafdc27f478095f0a9430220f390"
I20260809 18:55:30.939664 19474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9cb7b561f14cb6808136664219149d" candidate_uuid: "2c40e660d4724becbd468b2a095b604a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e089a9245d1b4aeebfe42c882bbe9a73"
I20260809 18:55:30.939721 19692 raft_consensus.cc:3037] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:30.939729 19474 raft_consensus.cc:3037] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:30.940374 19474 raft_consensus.cc:2454] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c40e660d4724becbd468b2a095b604a in term 1.
I20260809 18:55:30.940400 19692 raft_consensus.cc:2454] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c40e660d4724becbd468b2a095b604a in term 1.
I20260809 18:55:30.940538 19821 leader_election.cc:302] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c40e660d4724becbd468b2a095b604a, e089a9245d1b4aeebfe42c882bbe9a73; no voters:
I20260809 18:55:30.940625 20046 raft_consensus.cc:2781] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:30.940775 20046 raft_consensus.cc:686] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 1 LEADER]: Becoming Leader. State: Replica: 2c40e660d4724becbd468b2a095b604a, State: Running, Role: LEADER
I20260809 18:55:30.940881 20046 consensus_queue.cc:237] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } }
I20260809 18:55:30.941654 19234 catalog_manager.cc:5495] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c40e660d4724becbd468b2a095b604a (127.23.218.195). New cstate: current_term: 1 leader_uuid: "2c40e660d4724becbd468b2a095b604a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c40e660d4724becbd468b2a095b604a" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 37913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 } health_report { overall_health: UNKNOWN } } }
W20260809 18:55:30.952936 19545 tablet.cc:2369] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:55:30.973714 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:30.971795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.23.218.195:37913)
0809 18:55:30.971846 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:30.971848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:30.971856 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:55:30.971900 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:30.971901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:30.971916 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:30.971960 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:55:30.972063 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.972503 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.973153 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:30.973336 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:30.973391 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:30.973481 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:55:30.973570 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:55:30.973624 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:30.973637 (+ 13us) server_negotiation.cc:299] Negotiation successful
0809 18:55:30.973652 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:55:30.975299 19474 raft_consensus.cc:1261] T be9cb7b561f14cb6808136664219149d P e089a9245d1b4aeebfe42c882bbe9a73 [term 1 FOLLOWER]: Refusing update from remote peer 2c40e660d4724becbd468b2a095b604a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:30.975311 19692 raft_consensus.cc:1261] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [term 1 FOLLOWER]: Refusing update from remote peer 2c40e660d4724becbd468b2a095b604a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:30.975623 20052 consensus_queue.cc:1035] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c370cafdc27f478095f0a9430220f390" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 35979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:30.975749 20046 consensus_queue.cc:1035] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e089a9245d1b4aeebfe42c882bbe9a73" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:30.977837 20056 mvcc.cc:205] Tried to move back new op lower bound from 7316691890072657920 to 7316691889934442496. Current Snapshot: MvccSnapshot[applied={T|T < 7316691890072657920}]
I20260809 18:55:30.978091 20057 mvcc.cc:205] Tried to move back new op lower bound from 7316691890072657920 to 7316691889934442496. Current Snapshot: MvccSnapshot[applied={T|T < 7316691890072657920}]
I20260809 18:55:30.978801 20058 mvcc.cc:205] Tried to move back new op lower bound from 7316691890072657920 to 7316691889934442496. Current Snapshot: MvccSnapshot[applied={T|T < 7316691890072657920 or (T in {7316691890072657920})}]
I20260809 18:55:30.980316 19234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:30.980394 19234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:30.981076 19234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e62f7eb4e9a9473883a0298bc3e16da9: Sending DeleteTablet for 3 replicas of tablet be9cb7b561f14cb6808136664219149d
I20260809 18:55:30.981390 19893 tablet_service.cc:1514] Processing DeleteTablet for tablet be9cb7b561f14cb6808136664219149d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:30 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:35848
I20260809 18:55:30.981433 19659 tablet_service.cc:1514] Processing DeleteTablet for tablet be9cb7b561f14cb6808136664219149d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:30 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41394
I20260809 18:55:30.981607 19440 tablet_service.cc:1514] Processing DeleteTablet for tablet be9cb7b561f14cb6808136664219149d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:30 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:43152
I20260809 18:55:30.981706 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 19324
I20260809 18:55:30.993525 20076 tablet_replica.cc:330] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: stopping tablet replica
I20260809 18:55:30.993621 20076 raft_consensus.cc:2229] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:30.993702 20076 raft_consensus.cc:2258] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:30.993837 20075 tablet_replica.cc:330] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: stopping tablet replica
I20260809 18:55:30.993906 20075 raft_consensus.cc:2229] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:55:30.993969 20075 raft_consensus.cc:2258] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:30.996297 20075 ts_tablet_manager.cc:1905] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:30.997845 20075 ts_tablet_manager.cc:1918] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:30.997902 20075 log.cc:1196] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/be9cb7b561f14cb6808136664219149d
I20260809 18:55:30.998121 20075 ts_tablet_manager.cc:1939] T be9cb7b561f14cb6808136664219149d P 2c40e660d4724becbd468b2a095b604a: Deleting consensus metadata
I20260809 18:55:30.999805 19207 catalog_manager.cc:4841] TS 2c40e660d4724becbd468b2a095b604a (127.23.218.195:37913): tablet be9cb7b561f14cb6808136664219149d (table test-table [id=7b0ff66c35424c64986fe3f9c3945fbd]) successfully deleted
I20260809 18:55:31.000795 20076 ts_tablet_manager.cc:1905] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:31.002022 20076 ts_tablet_manager.cc:1918] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:31.002074 20076 log.cc:1196] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/be9cb7b561f14cb6808136664219149d
I20260809 18:55:31.002274 20076 ts_tablet_manager.cc:1939] T be9cb7b561f14cb6808136664219149d P c370cafdc27f478095f0a9430220f390: Deleting consensus metadata
I20260809 18:55:31.003293 19211 catalog_manager.cc:4841] TS c370cafdc27f478095f0a9430220f390 (127.23.218.194:35979): tablet be9cb7b561f14cb6808136664219149d (table test-table [id=7b0ff66c35424c64986fe3f9c3945fbd]) successfully deleted
W20260809 18:55:31.017076 19210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:40663 (error 108)
I20260809 18:55:31.017338 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 19550
I20260809 18:55:31.019721 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.017451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40663 (local address 127.0.0.1:43158)
0809 18:55:31.019650 (+ 2199us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.019694 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:40663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2177}
W20260809 18:55:31.019985 19210 catalog_manager.cc:4584] TS e089a9245d1b4aeebfe42c882bbe9a73 (127.23.218.193:40663): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be9cb7b561f14cb6808136664219149d: Network error: Client connection negotiation failed: client connection to 127.23.218.193:40663: connect: Connection refused (error 111)
I20260809 18:55:31.027019 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 19793
I20260809 18:55:31.053241 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 19183
2026-08-09T18:55:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2152 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T18:55:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:31Z Disabled control of system clock
I20260809 18:55:31.083458 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:37383
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:37717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:37383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:31.161448 20085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:31.161636 20085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:31.161662 20085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:31.162729 20085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:31.162760 20085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:31.162771 20085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:31.162782 20085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:31.162798 20085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:31.164057 20085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:37717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:37383
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:37383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:31.164232 20085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:31.164441 20085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:31.168710 20100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:31.168717 20094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:31.168738 20092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:31.168931 20085 server_base.cc:1034] running on GCE node
I20260809 18:55:31.169179 20085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:31.169431 20085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:31.170562 20085 hybrid_clock.cc:648] HybridClock initialized: now 1786301731170545 us; error 32 us; skew 500 ppm
I20260809 18:55:31.171684 20085 webserver.cc:460] Webserver started at http://127.23.218.254:36921/ using document root <none> and password file <none>
I20260809 18:55:31.171900 20085 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:31.171943 20085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:31.172065 20085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:31.173002 20085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "94d7b770d8874d54a1a983c8f1486d14"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "f88caaf9b2384f06d90721ece3fe499a"
server_key_iv: "5bbce4dc13ca0c54f2d02f312841d0d8"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.173303 20085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "94d7b770d8874d54a1a983c8f1486d14"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "f88caaf9b2384f06d90721ece3fe499a"
server_key_iv: "5bbce4dc13ca0c54f2d02f312841d0d8"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.174459 20085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:55:31.175113 20111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.175259 20085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:31.175336 20085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "94d7b770d8874d54a1a983c8f1486d14"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "f88caaf9b2384f06d90721ece3fe499a"
server_key_iv: "5bbce4dc13ca0c54f2d02f312841d0d8"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.175400 20085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:31.197376 20085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:31.197635 20085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:31.197767 20085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:31.201475 20085 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:37383
I20260809 18:55:31.201508 20194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:37383 every 8 connection(s)
I20260809 18:55:31.201814 20085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:31.202790 20195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:31.204962 20195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14: Bootstrap starting.
I20260809 18:55:31.205485 20195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:31.205755 20195 log.cc:824] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:31.206375 20195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14: No bootstrap required, opened a new log
I20260809 18:55:31.207266 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 20085
I20260809 18:55:31.207361 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:31.207535 24427 external_mini_cluster.cc:1428] Setting key d2a680d39812652cf32d0bc6c9d463b0
I20260809 18:55:31.208165 20195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7b770d8874d54a1a983c8f1486d14" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37383 } }
I20260809 18:55:31.208333 20195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:31.208376 20195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d7b770d8874d54a1a983c8f1486d14, State: Initialized, Role: FOLLOWER
I20260809 18:55:31.208495 20195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7b770d8874d54a1a983c8f1486d14" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37383 } }
I20260809 18:55:31.208571 20195 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:31.208659 20195 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:31.208732 20195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:31.209367 20195 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7b770d8874d54a1a983c8f1486d14" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37383 } }
I20260809 18:55:31.209470 20195 leader_election.cc:302] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94d7b770d8874d54a1a983c8f1486d14; no voters:
I20260809 18:55:31.209621 20195 leader_election.cc:288] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:31.209650 20203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:31.209810 20203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [term 1 LEADER]: Becoming Leader. State: Replica: 94d7b770d8874d54a1a983c8f1486d14, State: Running, Role: LEADER
I20260809 18:55:31.209847 20195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:31.209982 20203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7b770d8874d54a1a983c8f1486d14" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37383 } }
I20260809 18:55:31.210493 20207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94d7b770d8874d54a1a983c8f1486d14. Latest consensus state: current_term: 1 leader_uuid: "94d7b770d8874d54a1a983c8f1486d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7b770d8874d54a1a983c8f1486d14" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37383 } } }
I20260809 18:55:31.210579 20207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:31.211117 20218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:31.210790 20208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94d7b770d8874d54a1a983c8f1486d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7b770d8874d54a1a983c8f1486d14" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 37383 } } }
I20260809 18:55:31.211673 20218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:31.213351 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.207982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.23.218.254:37383)
0809 18:55:31.208215 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.208223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.208255 (+ 32us) server_negotiation.cc:407] Connection header received
0809 18:55:31.208360 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.208366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.208432 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.208567 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:31.208752 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.210417 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.211468 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.211961 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.212052 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.212665 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:31.212669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:31.212669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:31.212875 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:31.212917 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:31.212932 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:31.213001 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.213078 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.213155 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":101,"threads_started":1}
I20260809 18:55:31.213718 20208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:31.214247 20218 catalog_manager.cc:1347] Generated new cluster ID: 3038304c50e94045a47d41c7901f2839
I20260809 18:55:31.214303 20218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:31.231150 20218 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:31.231820 20218 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:31.244720 20218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14: Generated new TSK 0
I20260809 18:55:31.244889 20218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:31.252314 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37383
--builtin_ntp_servers=127.23.218.212:37717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:31.333428 20233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:31.333617 20233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:31.333647 20233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:31.334787 20233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:31.334826 20233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:31.334859 20233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:55:31.336182 20233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:37717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:31.336398 20233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:31.336648 20233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:31.341270 20246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:31.341282 20245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:31.341297 20251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:31.341425 20233 server_base.cc:1034] running on GCE node
I20260809 18:55:31.341689 20233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:31.341892 20233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:31.343037 20233 hybrid_clock.cc:648] HybridClock initialized: now 1786301731343026 us; error 30 us; skew 500 ppm
I20260809 18:55:31.344177 20233 webserver.cc:460] Webserver started at http://127.23.218.193:42247/ using document root <none> and password file <none>
I20260809 18:55:31.344375 20233 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:31.344419 20233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:31.344543 20233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:31.345574 20233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "815dbad3846345478cc1211e6d921601"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "384fe003921cb2ee89f0ed55a6163234"
server_key_iv: "6d9e6659503de0451a5b374291372333"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.345887 20233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "815dbad3846345478cc1211e6d921601"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "384fe003921cb2ee89f0ed55a6163234"
server_key_iv: "6d9e6659503de0451a5b374291372333"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.347112 20233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:31.347736 20259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.347886 20233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:31.347967 20233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "815dbad3846345478cc1211e6d921601"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "384fe003921cb2ee89f0ed55a6163234"
server_key_iv: "6d9e6659503de0451a5b374291372333"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.348028 20233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:31.362241 20233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:31.362504 20233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:31.362622 20233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:31.362818 20233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:31.363090 20233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:31.363121 20233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.363147 20233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:31.363171 20233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.368554 20233 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:40869
I20260809 18:55:31.368609 20451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:40869 every 8 connection(s)
I20260809 18:55:31.368875 20233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:55:31.373509 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.369460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:59495 (local address 127.23.218.254:37383)
0809 18:55:31.369500 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.369502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.369722 (+ 220us) server_negotiation.cc:407] Connection header received
0809 18:55:31.370164 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.370165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.370181 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.370233 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:31.370922 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.371402 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.372313 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.372502 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.372557 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.373257 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:31.373258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:31.373259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:31.373265 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:31.373281 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:31.373286 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:31.373330 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.373405 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.373435 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:55:31.373694 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.369446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:37383 (local address 127.23.218.193:59495)
0809 18:55:31.369621 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.369650 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:31.370073 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:31.370318 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:31.370322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:31.370332 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:31.370838 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.370843 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.371488 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.371489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.372225 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.372227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.372576 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.372577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.372703 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.373131 (+ 428us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:55:31.373132 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:31.373157 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:55:31.373179 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:31.373335 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:31.373340 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:31.373343 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:31.373404 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 18:55:31.373502 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260809 18:55:31.374125 20452 heartbeater.cc:347] Connected to a master server at 127.23.218.254:37383
I20260809 18:55:31.374202 20452 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:31.374393 20452 heartbeater.cc:511] Master 127.23.218.254:37383 requested a full tablet report, sending...
I20260809 18:55:31.374851 20139 ts_manager.cc:194] Registered new tserver with Master: 815dbad3846345478cc1211e6d921601 (127.23.218.193:40869)
I20260809 18:55:31.375437 20139 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.193:59495
I20260809 18:55:31.377509 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 20233
I20260809 18:55:31.377593 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:55:31.377765 24427 external_mini_cluster.cc:1428] Setting key 1265ca29b83698c4a3dac77f8c3c181e
I20260809 18:55:31.378553 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37383
--builtin_ntp_servers=127.23.218.212:37717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:31.457628 20459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:31.457805 20459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:31.457825 20459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:31.458949 20459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:31.458987 20459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:31.459012 20459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:55:31.460321 20459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:37717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:31.460530 20459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:31.460786 20459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:31.465351 20470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:31.465375 20468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:31.465351 20467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:31.465365 20459 server_base.cc:1034] running on GCE node
I20260809 18:55:31.465688 20459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:31.465886 20459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:31.467016 20459 hybrid_clock.cc:648] HybridClock initialized: now 1786301731467002 us; error 30 us; skew 500 ppm
I20260809 18:55:31.468111 20459 webserver.cc:460] Webserver started at http://127.23.218.194:38325/ using document root <none> and password file <none>
I20260809 18:55:31.468335 20459 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:31.468382 20459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:31.468504 20459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:31.469449 20459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "44723cd6434b4a588b6b361034ab9816"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "31149f75a14da8dc8ea4de1bffa0cd28"
server_key_iv: "a35cc48610d065b60a9b175f19cc887c"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.469765 20459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "44723cd6434b4a588b6b361034ab9816"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "31149f75a14da8dc8ea4de1bffa0cd28"
server_key_iv: "a35cc48610d065b60a9b175f19cc887c"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.470963 20459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:31.471580 20487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.471730 20459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:31.471808 20459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "44723cd6434b4a588b6b361034ab9816"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "31149f75a14da8dc8ea4de1bffa0cd28"
server_key_iv: "a35cc48610d065b60a9b175f19cc887c"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.471870 20459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:31.503338 20459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:31.503588 20459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:31.503715 20459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:31.503904 20459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:31.504189 20459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:31.504222 20459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.504249 20459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:31.504266 20459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.509815 20459 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:36583
I20260809 18:55:31.509855 20658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:36583 every 8 connection(s)
I20260809 18:55:31.510151 20459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:55:31.513393 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 20459
I20260809 18:55:31.513477 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:55:31.513664 24427 external_mini_cluster.cc:1428] Setting key 1b3eb55f8b6782f6a48ef431d58ae702
I20260809 18:55:31.514871 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37383
--builtin_ntp_servers=127.23.218.212:37717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 18:55:31.514981 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.510815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:50037 (local address 127.23.218.254:37383)
0809 18:55:31.510860 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.510862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.511081 (+ 219us) server_negotiation.cc:407] Connection header received
0809 18:55:31.511514 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.511515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.511525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.511576 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:31.512197 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.512673 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.513598 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.513842 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.513896 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.514718 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:31.514720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:31.514720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:31.514725 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:31.514742 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:31.514746 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:31.514793 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.514874 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.514904 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:55:31.515162 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.510803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:37383 (local address 127.23.218.194:50037)
0809 18:55:31.510972 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.511002 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:31.511436 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:31.511645 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:31.511648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:31.511656 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:31.512135 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.512140 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.512737 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.512737 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.513514 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.513517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.513927 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.513928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.514075 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.514578 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:55:31.514579 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:31.514606 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:55:31.514638 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:31.514799 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:31.514803 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:31.514806 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:31.514870 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 18:55:31.514967 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260809 18:55:31.515602 20660 heartbeater.cc:347] Connected to a master server at 127.23.218.254:37383
I20260809 18:55:31.515687 20660 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:31.515853 20660 heartbeater.cc:511] Master 127.23.218.254:37383 requested a full tablet report, sending...
I20260809 18:55:31.516191 20139 ts_manager.cc:194] Registered new tserver with Master: 44723cd6434b4a588b6b361034ab9816 (127.23.218.194:36583)
I20260809 18:55:31.516638 20139 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.194:50037
W20260809 18:55:31.595069 20664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:31.595257 20664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:31.595278 20664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:31.596315 20664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:31.596346 20664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:31.596369 20664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:55:31.597628 20664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:37717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:37383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:31.597795 20664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:31.598021 20664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:31.602627 20675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:31.602706 20664 server_base.cc:1034] running on GCE node
W20260809 18:55:31.602628 20671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:31.602716 20670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:31.603020 20664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:31.603237 20664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:31.604398 20664 hybrid_clock.cc:648] HybridClock initialized: now 1786301731604380 us; error 34 us; skew 500 ppm
I20260809 18:55:31.605489 20664 webserver.cc:460] Webserver started at http://127.23.218.195:39221/ using document root <none> and password file <none>
I20260809 18:55:31.605690 20664 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:31.605728 20664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:31.605816 20664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:31.606794 20664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "c337171bb0f14fb0b5a55b8654724308"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "4d29652bab93d8e5a7dc6e2e52964fe2"
server_key_iv: "cb70dcff06b6fd7ae4f3c3b6f56baa49"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.607074 20664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "c337171bb0f14fb0b5a55b8654724308"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "4d29652bab93d8e5a7dc6e2e52964fe2"
server_key_iv: "cb70dcff06b6fd7ae4f3c3b6f56baa49"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.608160 20664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:55:31.608743 20685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.608879 20664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:31.608961 20664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "c337171bb0f14fb0b5a55b8654724308"
format_stamp: "Formatted at 2026-08-09 18:55:31 on dist-test-slave-6zbq"
server_key: "4d29652bab93d8e5a7dc6e2e52964fe2"
server_key_iv: "cb70dcff06b6fd7ae4f3c3b6f56baa49"
server_key_version: "encryptionkey@0"
I20260809 18:55:31.609026 20664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:31.628677 20664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:31.628901 20664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:31.629010 20664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:31.629199 20664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:31.629480 20664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:31.629513 20664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.629539 20664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:31.629562 20664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:31.634842 20664 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:38577
I20260809 18:55:31.634889 20891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:38577 every 8 connection(s)
I20260809 18:55:31.635162 20664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:55:31.639652 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.635834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:51733 (local address 127.23.218.254:37383)
0809 18:55:31.635895 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.635897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.636110 (+ 213us) server_negotiation.cc:407] Connection header received
0809 18:55:31.636498 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.636499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.636510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.636565 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:31.637133 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.637621 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.638540 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.638736 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.638789 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.639431 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:31.639432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:31.639432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:31.639437 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:31.639452 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:31.639456 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:31.639502 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.639562 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.639590 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 18:55:31.639806 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.635792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:37383 (local address 127.23.218.195:51733)
0809 18:55:31.635962 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.635998 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:31.636424 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:31.636612 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:31.636615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:31.636622 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:31.637074 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.637078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.637690 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.637690 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.638462 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.638465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.638798 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.638798 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.638895 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.639297 (+ 402us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:55:31.639298 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:31.639322 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:55:31.639344 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:31.639503 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:31.639505 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:31.639507 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:55:31.639549 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 18:55:31.639619 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260809 18:55:31.640112 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 20664
I20260809 18:55:31.640219 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:55:31.640322 20892 heartbeater.cc:347] Connected to a master server at 127.23.218.254:37383
I20260809 18:55:31.640407 20892 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:31.640411 24427 external_mini_cluster.cc:1428] Setting key 67034f0181b9f2cf8df6440478bc65c8
I20260809 18:55:31.640591 20892 heartbeater.cc:511] Master 127.23.218.254:37383 requested a full tablet report, sending...
I20260809 18:55:31.640945 20139 ts_manager.cc:194] Registered new tserver with Master: c337171bb0f14fb0b5a55b8654724308 (127.23.218.195:38577)
I20260809 18:55:31.641292 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:55:31.641341 20139 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.195:51733
I20260809 18:55:31.645213 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.642615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41382 (local address 127.23.218.254:37383)
0809 18:55:31.642652 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.642654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.642778 (+ 124us) server_negotiation.cc:407] Connection header received
0809 18:55:31.642827 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.642828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.642839 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.642883 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:31.643033 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.643480 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.644189 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.644383 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.644434 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.645002 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:31.645003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:31.645004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:31.645008 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:31.645023 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:31.645027 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:31.645071 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.645123 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.645146 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:55:31.646551 20139 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:55:31.647991 20139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:55:31.653728 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.649902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50524 (local address 127.23.218.195:38577)
0809 18:55:31.650062 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.650066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.650076 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:55:31.650106 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.650108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.650120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.650207 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:31.650357 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.651284 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.652770 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.653365 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.653426 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.653507 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.653619 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.653653 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260809 18:55:31.653723 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.649833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:38577 (local address 127.0.0.1:50524)
0809 18:55:31.649946 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.649973 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:31.650034 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:31.650212 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:31.650214 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:31.650216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:31.650275 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.650276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.651396 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.651397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.652678 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.652681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.653445 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.653446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.653523 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.653536 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 18:55:31.653605 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 18:55:31.653654 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260809 18:55:31.653903 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.650560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.23.218.193:40869)
0809 18:55:31.650748 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.650752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.650761 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:55:31.650797 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.650799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.650813 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.650886 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:31.651056 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.651838 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.652802 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.653394 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.653460 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.653535 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.653783 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.653822 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
I20260809 18:55:31.653942 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.650162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40869 (local address 127.0.0.1:41072)
0809 18:55:31.650409 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.650418 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:31.650458 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:31.650947 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:31.650948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:31.650949 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:31.650981 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.650982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.651913 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.651914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.652742 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.652745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.653572 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.653573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.653635 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.653639 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:31.653700 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 18:55:31.653740 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":36,"thread_start_us":58,"threads_started":1}
I20260809 18:55:31.654182 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.650023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:36583 (local address 127.0.0.1:45044)
0809 18:55:31.650126 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.650135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:31.650179 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:31.650758 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:31.650759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:31.650762 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:31.650840 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.650842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.651933 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.651934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.653005 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.653008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.653957 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.653958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.654025 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.654029 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:31.654084 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 18:55:31.654120 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":59,"threads_started":1}
I20260809 18:55:31.654240 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.650379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45044 (local address 127.23.218.194:36583)
0809 18:55:31.650527 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.650532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.650542 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:55:31.650580 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.650582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.650598 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.650675 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:31.650918 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.651842 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.653085 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.653788 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.653846 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.653921 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.654110 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.654160 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260809 18:55:31.654642 20345 tablet_service.cc:1467] Processing CreateTablet for tablet 14a3fb81bf194131addc0ee4d60393dc (DEFAULT_TABLE table=test-table [id=a2ee42b5c6c44c33941b4ebe53599589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:31.655006 20345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a3fb81bf194131addc0ee4d60393dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:31.654934 20555 tablet_service.cc:1467] Processing CreateTablet for tablet 14a3fb81bf194131addc0ee4d60393dc (DEFAULT_TABLE table=test-table [id=a2ee42b5c6c44c33941b4ebe53599589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:31.655231 20555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a3fb81bf194131addc0ee4d60393dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:31.657971 20777 tablet_service.cc:1467] Processing CreateTablet for tablet 14a3fb81bf194131addc0ee4d60393dc (DEFAULT_TABLE table=test-table [id=a2ee42b5c6c44c33941b4ebe53599589]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:31.658223 20777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14a3fb81bf194131addc0ee4d60393dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:31.658488 20924 tablet_bootstrap.cc:492] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601: Bootstrap starting.
I20260809 18:55:31.659124 20924 tablet_bootstrap.cc:654] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:31.659430 20924 log.cc:824] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:31.658293 20923 tablet_bootstrap.cc:492] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: Bootstrap starting.
I20260809 18:55:31.660069 20924 tablet_bootstrap.cc:492] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601: No bootstrap required, opened a new log
I20260809 18:55:31.660115 20924 ts_tablet_manager.cc:1397] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:31.660244 20923 tablet_bootstrap.cc:654] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:31.660545 20923 log.cc:824] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:31.661540 20923 tablet_bootstrap.cc:492] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: No bootstrap required, opened a new log
I20260809 18:55:31.661549 20928 tablet_bootstrap.cc:492] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: Bootstrap starting.
I20260809 18:55:31.661610 20923 ts_tablet_manager.cc:1397] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 18:55:31.662132 20928 tablet_bootstrap.cc:654] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:31.662258 20924 raft_consensus.cc:348] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } }
I20260809 18:55:31.662425 20924 raft_consensus.cc:374] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:31.662482 20924 raft_consensus.cc:729] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815dbad3846345478cc1211e6d921601, State: Initialized, Role: FOLLOWER
I20260809 18:55:31.662511 20928 log.cc:824] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:31.662603 20924 consensus_queue.cc:260] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } }
I20260809 18:55:31.662864 20924 ts_tablet_manager.cc:1428] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:55:31.662896 20452 heartbeater.cc:503] Master 127.23.218.254:37383 was elected leader, sending a full tablet report...
I20260809 18:55:31.663157 20928 tablet_bootstrap.cc:492] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: No bootstrap required, opened a new log
I20260809 18:55:31.663240 20928 ts_tablet_manager.cc:1397] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:31.664192 20923 raft_consensus.cc:348] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } }
I20260809 18:55:31.664351 20923 raft_consensus.cc:374] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:31.664393 20923 raft_consensus.cc:729] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44723cd6434b4a588b6b361034ab9816, State: Initialized, Role: FOLLOWER
I20260809 18:55:31.664503 20923 consensus_queue.cc:260] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } }
I20260809 18:55:31.664752 20923 ts_tablet_manager.cc:1428] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:55:31.664786 20660 heartbeater.cc:503] Master 127.23.218.254:37383 was elected leader, sending a full tablet report...
I20260809 18:55:31.665459 20928 raft_consensus.cc:348] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } }
I20260809 18:55:31.665606 20928 raft_consensus.cc:374] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:31.665653 20928 raft_consensus.cc:729] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c337171bb0f14fb0b5a55b8654724308, State: Initialized, Role: FOLLOWER
I20260809 18:55:31.665752 20928 consensus_queue.cc:260] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } }
I20260809 18:55:31.665934 20928 ts_tablet_manager.cc:1428] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:55:31.666000 20892 heartbeater.cc:503] Master 127.23.218.254:37383 was elected leader, sending a full tablet report...
W20260809 18:55:31.760658 20661 tablet.cc:2369] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:55:31.869457 20455 tablet.cc:2369] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:55:31.885675 20894 tablet.cc:2369] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:55:31.889612 20932 raft_consensus.cc:482] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:31.889712 20932 raft_consensus.cc:504] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } }
I20260809 18:55:31.890003 20932 leader_election.cc:288] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c337171bb0f14fb0b5a55b8654724308 (127.23.218.195:38577), 44723cd6434b4a588b6b361034ab9816 (127.23.218.194:36583)
I20260809 18:55:31.893755 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.890248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:51789 (local address 127.23.218.195:38577)
0809 18:55:31.890308 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.890311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.890320 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:55:31.890437 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.890438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.890449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.890477 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:31.890714 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.891421 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.892842 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.893392 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.893463 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.893471 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.893641 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.893682 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 18:55:31.893750 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.890393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:36817 (local address 127.23.218.194:36583)
0809 18:55:31.890437 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.890439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.890449 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:55:31.890495 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.890496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.890503 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.890547 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:31.890688 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.891452 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.892843 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.893392 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.893463 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.893471 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.893641 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.893682 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:31.893769 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.890144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:38577 (local address 127.23.218.193:51789)
0809 18:55:31.890249 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.890259 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:31.890323 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:31.890547 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:31.890548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:31.890550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:31.890633 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.890635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.891493 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.891494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.892759 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.892763 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.893471 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.893472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.893570 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.893573 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:31.893626 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 18:55:31.893670 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":49,"thread_start_us":49,"threads_started":1}
I20260809 18:55:31.893769 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.890295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:36583 (local address 127.23.218.193:36817)
0809 18:55:31.890393 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.890402 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:31.890442 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:31.890562 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:31.890562 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:31.890563 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:31.890626 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.890629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.891514 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.891515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.892759 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:31.892763 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.893470 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.893471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:31.893568 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.893572 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:31.893626 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 18:55:31.893665 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":85,"thread_start_us":60,"threads_started":1}
I20260809 18:55:31.894104 20805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a3fb81bf194131addc0ee4d60393dc" candidate_uuid: "815dbad3846345478cc1211e6d921601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c337171bb0f14fb0b5a55b8654724308" is_pre_election: true
I20260809 18:55:31.894156 20587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a3fb81bf194131addc0ee4d60393dc" candidate_uuid: "815dbad3846345478cc1211e6d921601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44723cd6434b4a588b6b361034ab9816" is_pre_election: true
I20260809 18:55:31.894239 20805 raft_consensus.cc:2454] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815dbad3846345478cc1211e6d921601 in term 0.
I20260809 18:55:31.894321 20587 raft_consensus.cc:2454] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815dbad3846345478cc1211e6d921601 in term 0.
I20260809 18:55:31.894493 20264 leader_election.cc:302] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815dbad3846345478cc1211e6d921601, c337171bb0f14fb0b5a55b8654724308; no voters:
I20260809 18:55:31.894629 20932 raft_consensus.cc:2781] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:55:31.894703 20932 raft_consensus.cc:482] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:55:31.894719 20932 raft_consensus.cc:3037] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:31.895433 20932 raft_consensus.cc:504] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } }
I20260809 18:55:31.895546 20932 leader_election.cc:288] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [CANDIDATE]: Term 1 election: Requested vote from peers c337171bb0f14fb0b5a55b8654724308 (127.23.218.195:38577), 44723cd6434b4a588b6b361034ab9816 (127.23.218.194:36583)
I20260809 18:55:31.895694 20805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a3fb81bf194131addc0ee4d60393dc" candidate_uuid: "815dbad3846345478cc1211e6d921601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c337171bb0f14fb0b5a55b8654724308"
I20260809 18:55:31.895711 20587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14a3fb81bf194131addc0ee4d60393dc" candidate_uuid: "815dbad3846345478cc1211e6d921601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44723cd6434b4a588b6b361034ab9816"
I20260809 18:55:31.895754 20805 raft_consensus.cc:3037] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:31.895783 20587 raft_consensus.cc:3037] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:31.896437 20805 raft_consensus.cc:2454] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815dbad3846345478cc1211e6d921601 in term 1.
I20260809 18:55:31.896486 20587 raft_consensus.cc:2454] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815dbad3846345478cc1211e6d921601 in term 1.
I20260809 18:55:31.896617 20264 leader_election.cc:302] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 815dbad3846345478cc1211e6d921601, c337171bb0f14fb0b5a55b8654724308; no voters:
I20260809 18:55:31.896725 20932 raft_consensus.cc:2781] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:31.896880 20932 raft_consensus.cc:686] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [term 1 LEADER]: Becoming Leader. State: Replica: 815dbad3846345478cc1211e6d921601, State: Running, Role: LEADER
I20260809 18:55:31.896978 20932 consensus_queue.cc:237] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } }
I20260809 18:55:31.897760 20139 catalog_manager.cc:5495] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 reported cstate change: term changed from 0 to 1, leader changed from <none> to 815dbad3846345478cc1211e6d921601 (127.23.218.193). New cstate: current_term: 1 leader_uuid: "815dbad3846345478cc1211e6d921601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "815dbad3846345478cc1211e6d921601" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40869 } health_report { overall_health: HEALTHY } } }
I20260809 18:55:31.932533 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.930301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.23.218.193:40869)
0809 18:55:31.930377 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:31.930379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:31.930387 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:55:31.930425 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:31.930426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:31.930447 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:31.930509 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:55:31.930630 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.931100 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.931827 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:31.932030 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:31.932084 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:31.932231 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:55:31.932336 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:55:31.932407 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:31.932442 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 18:55:31.932462 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 18:55:31.934211 20805 raft_consensus.cc:1261] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [term 1 FOLLOWER]: Refusing update from remote peer 815dbad3846345478cc1211e6d921601: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:31.934217 20587 raft_consensus.cc:1261] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [term 1 FOLLOWER]: Refusing update from remote peer 815dbad3846345478cc1211e6d921601: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:31.934535 20937 consensus_queue.cc:1035] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44723cd6434b4a588b6b361034ab9816" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 36583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:31.934661 20932 consensus_queue.cc:1035] T 14a3fb81bf194131addc0ee4d60393dc P 815dbad3846345478cc1211e6d921601 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c337171bb0f14fb0b5a55b8654724308" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:31.937417 20945 mvcc.cc:205] Tried to move back new op lower bound from 7316691894000099328 to 7316691893850628096. Current Snapshot: MvccSnapshot[applied={T|T < 7316691894000099328}]
I20260809 18:55:31.939119 20139 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:31.939198 20139 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:31.939870 20139 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94d7b770d8874d54a1a983c8f1486d14: Sending DeleteTablet for 3 replicas of tablet 14a3fb81bf194131addc0ee4d60393dc
I20260809 18:55:31.940186 20777 tablet_service.cc:1514] Processing DeleteTablet for tablet 14a3fb81bf194131addc0ee4d60393dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:31 UTC) from {username='slave'} at 127.0.0.1:50524
I20260809 18:55:31.940235 20345 tablet_service.cc:1514] Processing DeleteTablet for tablet 14a3fb81bf194131addc0ee4d60393dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:31 UTC) from {username='slave'} at 127.0.0.1:41072
I20260809 18:55:31.940248 20555 tablet_service.cc:1514] Processing DeleteTablet for tablet 14a3fb81bf194131addc0ee4d60393dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:31 UTC) from {username='slave'} at 127.0.0.1:45044
I20260809 18:55:31.940476 20960 tablet_replica.cc:330] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: stopping tablet replica
I20260809 18:55:31.940598 20960 raft_consensus.cc:2229] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:31.940627 20958 tablet_replica.cc:330] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: stopping tablet replica
I20260809 18:55:31.940760 20958 raft_consensus.cc:2229] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:31.940783 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 20233
I20260809 18:55:31.940783 20960 raft_consensus.cc:2258] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:31.946501 20958 raft_consensus.cc:2258] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:31.947114 20958 ts_tablet_manager.cc:1905] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:31.948974 20960 ts_tablet_manager.cc:1905] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:31.949178 20958 ts_tablet_manager.cc:1918] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:31.949270 20958 log.cc:1196] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/14a3fb81bf194131addc0ee4d60393dc
I20260809 18:55:31.949589 20958 ts_tablet_manager.cc:1939] T 14a3fb81bf194131addc0ee4d60393dc P c337171bb0f14fb0b5a55b8654724308: Deleting consensus metadata
I20260809 18:55:31.950086 20115 catalog_manager.cc:4841] TS c337171bb0f14fb0b5a55b8654724308 (127.23.218.195:38577): tablet 14a3fb81bf194131addc0ee4d60393dc (table test-table [id=a2ee42b5c6c44c33941b4ebe53599589]) successfully deleted
I20260809 18:55:31.950254 20960 ts_tablet_manager.cc:1918] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:31.950320 20960 log.cc:1196] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/14a3fb81bf194131addc0ee4d60393dc
I20260809 18:55:31.950527 20960 ts_tablet_manager.cc:1939] T 14a3fb81bf194131addc0ee4d60393dc P 44723cd6434b4a588b6b361034ab9816: Deleting consensus metadata
I20260809 18:55:31.950909 20115 catalog_manager.cc:4841] TS 44723cd6434b4a588b6b361034ab9816 (127.23.218.194:36583): tablet 14a3fb81bf194131addc0ee4d60393dc (table test-table [id=a2ee42b5c6c44c33941b4ebe53599589]) successfully deleted
W20260809 18:55:31.954754 20115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:40869 (error 108)
I20260809 18:55:31.955004 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 20459
I20260809 18:55:31.955178 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:31.955051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40869 (local address 127.0.0.1:41096)
0809 18:55:31.955101 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:31.955141 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:40869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 18:55:31.956651 20115 catalog_manager.cc:4584] TS 815dbad3846345478cc1211e6d921601 (127.23.218.193:40869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14a3fb81bf194131addc0ee4d60393dc: Network error: Client connection negotiation failed: client connection to 127.23.218.193:40869: connect: Connection refused (error 111)
I20260809 18:55:31.968886 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 20664
I20260809 18:55:31.980121 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 20085
2026-08-09T18:55:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (926 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:32 dist-test-slave-6zbq krb5kdc[20966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:32 dist-test-slave-6zbq krb5kdc[20966](info): set up 2 sockets
Aug 09 18:55:32 dist-test-slave-6zbq krb5kdc[20966](info): commencing operation
krb5kdc: starting...
W20260809 18:55:33.116657 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:33 dist-test-slave-6zbq krb5kdc[20966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:55:33.224546 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:33107
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:43751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:33107
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:33.302121 20985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:33.302300 20985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:33.302320 20985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:33.303434 20985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:33.303472 20985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:33.303490 20985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:33.303505 20985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:33.303525 20985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:33.304934 20985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:43751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:33107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:33107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:33.305109 20985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:33.305269 20985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:33.309578 20993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:33.309571 20997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:33.309630 20985 server_base.cc:1034] running on GCE node
W20260809 18:55:33.309631 20995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:33.310066 20985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:33.310329 20985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:33.311486 20985 hybrid_clock.cc:648] HybridClock initialized: now 1786301733311473 us; error 29 us; skew 500 ppm
Aug 09 18:55:33 dist-test-slave-6zbq krb5kdc[20966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301733, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:33.313762 20985 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:55:33.314137 20985 webserver.cc:460] Webserver started at http://127.23.218.254:34719/ using document root <none> and password file <none>
I20260809 18:55:33.314371 20985 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:33.314422 20985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:33.314544 20985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:33.315493 20985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "8cd1767ca60b3661e488ecb05596900d"
server_key_iv: "ea7f23971bdd1e366c2b1120709a112f"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.315799 20985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "8cd1767ca60b3661e488ecb05596900d"
server_key_iv: "ea7f23971bdd1e366c2b1120709a112f"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.316959 20985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:55:33.317559 21016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.317703 20985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:55:33.317785 20985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "8cd1767ca60b3661e488ecb05596900d"
server_key_iv: "ea7f23971bdd1e366c2b1120709a112f"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.317837 20985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:33.330081 20985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:33.331646 20985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:33.331743 20985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:33.335318 20985 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:33107
I20260809 18:55:33.335358 21097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:33107 every 8 connection(s)
I20260809 18:55:33.335652 20985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:33.336421 21098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:33.338304 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 20985
I20260809 18:55:33.338449 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:33.338515 21098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671: Bootstrap starting.
I20260809 18:55:33.338629 24427 external_mini_cluster.cc:1428] Setting key a6fb5c568c211c4bcea2c69a7fbcba27
I20260809 18:55:33.339177 21098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:33.339473 21098 log.cc:824] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:33.340107 21098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671: No bootstrap required, opened a new log
I20260809 18:55:33.342152 21098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33107 } }
I20260809 18:55:33.342307 21098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:33.342360 21098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d5cfbc8a1e42e7bcbbda6f92aea671, State: Initialized, Role: FOLLOWER
I20260809 18:55:33.342474 21098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33107 } }
I20260809 18:55:33.342547 21098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:33.342584 21098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:33.342634 21098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:33.343251 21098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33107 } }
I20260809 18:55:33.343355 21098 leader_election.cc:302] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60d5cfbc8a1e42e7bcbbda6f92aea671; no voters:
I20260809 18:55:33.343508 21098 leader_election.cc:288] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:33.343525 21104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:33.343629 21104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [term 1 LEADER]: Becoming Leader. State: Replica: 60d5cfbc8a1e42e7bcbbda6f92aea671, State: Running, Role: LEADER
I20260809 18:55:33.343757 21098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:33.343766 21104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33107 } }
I20260809 18:55:33.344184 21108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60d5cfbc8a1e42e7bcbbda6f92aea671. Latest consensus state: current_term: 1 leader_uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33107 } } }
I20260809 18:55:33.344339 21108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:33.344826 21106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d5cfbc8a1e42e7bcbbda6f92aea671" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33107 } } }
I20260809 18:55:33.344909 21106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671 [sys.catalog]: This master's current role is: LEADER
Aug 09 18:55:33 dist-test-slave-6zbq krb5kdc[20966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
W20260809 18:55:33.345574 21128 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:55:33.345630 21128 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:55:33.345683 21119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:33.345870 21119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:33.347740 21119 catalog_manager.cc:1347] Generated new cluster ID: ede9fdf2b7df4f4fb4348a66443b26cc
I20260809 18:55:33.347790 21119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:33.349151 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.339081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.23.218.254:33107)
0809 18:55:33.339359 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.339366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.339389 (+ 23us) server_negotiation.cc:407] Connection header received
0809 18:55:33.339821 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.339828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.339873 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.339980 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:33.340718 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.341564 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.342622 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.343200 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.343333 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.345927 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:33.345931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:33.345931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:33.345942 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:33.347597 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.347990 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.347993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.347993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.348100 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.348292 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.348293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.348294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.348582 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:33.348728 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.348816 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.348952 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":59,"threads_started":1}
I20260809 18:55:33.354606 21119 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:33.355067 21119 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:33.358098 21119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671: Generated new TSK 0
I20260809 18:55:33.358247 21119 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:55:33.386139 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33107
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:43751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:33.500592 21135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:33.500829 21135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:33.500864 21135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:33.502429 21135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:33.502480 21135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:33.502512 21135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:55:33.504405 21135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:43751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:33.504643 21135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:33.504884 21135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:33.509452 21147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:33.509567 21135 server_base.cc:1034] running on GCE node
W20260809 18:55:33.509433 21144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:33.509418 21143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:33.509862 21135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:33.510066 21135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:33.511241 21135 hybrid_clock.cc:648] HybridClock initialized: now 1786301733511227 us; error 32 us; skew 500 ppm
Aug 09 18:55:33 dist-test-slave-6zbq krb5kdc[20966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301733, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:33.513602 21135 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:55:33.513980 21135 webserver.cc:460] Webserver started at http://127.23.218.193:39057/ using document root <none> and password file <none>
I20260809 18:55:33.514183 21135 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:33.514226 21135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:33.514358 21135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:33.515311 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "606e539c9cd746a0a650a2987b799d1b"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "690918f86f514a57d6ed2c178a7e7814"
server_key_iv: "fef6f449a472899434d294b977435713"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.515621 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "606e539c9cd746a0a650a2987b799d1b"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "690918f86f514a57d6ed2c178a7e7814"
server_key_iv: "fef6f449a472899434d294b977435713"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.516788 21135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:55:33.517409 21160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.517540 21135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:33.517622 21135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "606e539c9cd746a0a650a2987b799d1b"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "690918f86f514a57d6ed2c178a7e7814"
server_key_iv: "fef6f449a472899434d294b977435713"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.517683 21135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:33.532531 21135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:33.534478 21135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:33.534603 21135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:33.534786 21135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:33.535041 21135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:33.535072 21135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.535109 21135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:33.535133 21135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.540349 21135 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:45091
I20260809 18:55:33.540385 21336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:45091 every 8 connection(s)
I20260809 18:55:33.540639 21135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:55:33.540838 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 21135
I20260809 18:55:33.540942 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:55:33.541152 24427 external_mini_cluster.cc:1428] Setting key 432332d2457b607dfcc7063da054523e
Aug 09 18:55:33 dist-test-slave-6zbq krb5kdc[20966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301733, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:33.550221 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.541298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:35777 (local address 127.23.218.254:33107)
0809 18:55:33.541340 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.541342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.541579 (+ 237us) server_negotiation.cc:407] Connection header received
0809 18:55:33.542128 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.542129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.542138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.542185 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:33.543285 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.543821 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.545097 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.545400 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.545467 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.547701 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:33.547702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:33.547703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:33.547709 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:33.548911 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.549358 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.549359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.549359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.549406 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.549655 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.549655 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.549656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.549745 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:33.549789 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.550096 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.550161 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:55:33.550483 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.541278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:33107 (local address 127.23.218.193:35777)
0809 18:55:33.541453 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:33.541497 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:33.542029 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:33.542249 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:33.542253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:33.542953 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:33.543202 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.543208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.543914 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.543915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.545013 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.545017 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.545481 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.545482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.545590 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.545991 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:33.545998 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:33.547611 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:33.548993 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:33.549001 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:33.549003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:33.549292 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:33.549459 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:33.549460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:33.549461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:33.549597 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:33.549799 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:33.549805 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:33.549958 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 18:55:33.550094 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 18:55:33.550255 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
I20260809 18:55:33.550982 21337 heartbeater.cc:347] Connected to a master server at 127.23.218.254:33107
I20260809 18:55:33.551079 21337 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:33.551234 21337 heartbeater.cc:511] Master 127.23.218.254:33107 requested a full tablet report, sending...
I20260809 18:55:33.551666 21041 ts_manager.cc:194] Registered new tserver with Master: 606e539c9cd746a0a650a2987b799d1b (127.23.218.193:45091)
I20260809 18:55:33.552228 21041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:35777
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:55:33.564487 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33107
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:43751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:33.642684 21344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:33.642854 21344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:33.642874 21344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:33.643985 21344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:33.644023 21344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:33.644052 21344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:55:33.645561 21344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:43751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:33.645769 21344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:33.645967 21344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:33.650480 21354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:33.650559 21344 server_base.cc:1034] running on GCE node
W20260809 18:55:33.650465 21353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:33.650456 21357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:33.651062 21344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:33.651284 21344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:33.652417 21344 hybrid_clock.cc:648] HybridClock initialized: now 1786301733652400 us; error 28 us; skew 500 ppm
Aug 09 18:55:33 dist-test-slave-6zbq krb5kdc[20966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301733, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:33.654757 21344 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:55:33.655109 21344 webserver.cc:460] Webserver started at http://127.23.218.194:42559/ using document root <none> and password file <none>
I20260809 18:55:33.655267 21344 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:33.655305 21344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:33.655392 21344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:33.656288 21344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "234afbf6adf84d099af841a5a0f4be3a"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "e3ae435639b7b1d24fc4d1e9ed0cf7e7"
server_key_iv: "2f520e141464e4a460fe340d2619f325"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.656559 21344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "234afbf6adf84d099af841a5a0f4be3a"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "e3ae435639b7b1d24fc4d1e9ed0cf7e7"
server_key_iv: "2f520e141464e4a460fe340d2619f325"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.657752 21344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:55:33.658362 21368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.658514 21344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.658597 21344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "234afbf6adf84d099af841a5a0f4be3a"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "e3ae435639b7b1d24fc4d1e9ed0cf7e7"
server_key_iv: "2f520e141464e4a460fe340d2619f325"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.658666 21344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:33.690946 21344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:33.692480 21344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:33.692585 21344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:33.692782 21344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:33.693035 21344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:33.693068 21344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.693085 21344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:33.693109 21344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.698364 21344 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:33419
I20260809 18:55:33.698432 21543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:33419 every 8 connection(s)
I20260809 18:55:33.698678 21344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
Aug 09 18:55:33 dist-test-slave-6zbq krb5kdc[20966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301733, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:33.706564 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.699292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:36145 (local address 127.23.218.254:33107)
0809 18:55:33.699371 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.699374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.699533 (+ 159us) server_negotiation.cc:407] Connection header received
0809 18:55:33.699961 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.699962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.699972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.700015 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:33.700860 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.701350 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.702272 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.702503 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.702554 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.704435 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:33.704436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:33.704436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:33.704441 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:33.705464 (+ 1023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.705787 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.705787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.705788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.705830 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.705997 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.705997 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.705998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.706084 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:33.706126 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.706425 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.706485 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 18:55:33.706740 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.699262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:33107 (local address 127.23.218.194:36145)
0809 18:55:33.699414 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:33.699446 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:33.699883 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:33.700072 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:33.700074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:33.700593 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:33.700792 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.700797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.701415 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.701416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.702207 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.702209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.702563 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.702564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.702659 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.703098 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:33.703106 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:33.704368 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:33.705524 (+ 1156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:33.705531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:33.705532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:33.705732 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:33.705873 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:33.705873 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:33.705873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:33.705957 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:33.706126 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:33.706129 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:33.706281 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 18:55:33.706407 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 18:55:33.706536 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260809 18:55:33.707186 21545 heartbeater.cc:347] Connected to a master server at 127.23.218.254:33107
I20260809 18:55:33.707265 21545 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:33.707424 21545 heartbeater.cc:511] Master 127.23.218.254:33107 requested a full tablet report, sending...
I20260809 18:55:33.707767 21041 ts_manager.cc:194] Registered new tserver with Master: 234afbf6adf84d099af841a5a0f4be3a (127.23.218.194:33419)
I20260809 18:55:33.708110 21041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:36145
I20260809 18:55:33.708806 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 21344
I20260809 18:55:33.708904 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:55:33.709059 24427 external_mini_cluster.cc:1428] Setting key c984697c139d9bf865eefbc3c726ddcd
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:55:33.730136 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33107
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:43751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:33.808014 21552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:33.808190 21552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:33.808212 21552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:33.809312 21552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:33.809353 21552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:33.809382 21552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:55:33.810830 21552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:43751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:33107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:33.811029 21552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:33.811219 21552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:33.815735 21559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:33.815866 21552 server_base.cc:1034] running on GCE node
W20260809 18:55:33.815737 21562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:33.816077 21565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:33.816282 21552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:33.816488 21552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:33.817628 21552 hybrid_clock.cc:648] HybridClock initialized: now 1786301733817615 us; error 28 us; skew 500 ppm
Aug 09 18:55:33 dist-test-slave-6zbq krb5kdc[20966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301733, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:33.819865 21552 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:55:33.820297 21552 webserver.cc:460] Webserver started at http://127.23.218.195:36133/ using document root <none> and password file <none>
I20260809 18:55:33.820504 21552 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:33.820549 21552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:33.820669 21552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:33.821771 21552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "ae1b9f97f045489083a233037d7f03a4"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "7fbbfb91235886931dae6a85bb35b3fe"
server_key_iv: "2302c1f6a2811bb6987c8dd88dc37e9e"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.822088 21552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "ae1b9f97f045489083a233037d7f03a4"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "7fbbfb91235886931dae6a85bb35b3fe"
server_key_iv: "2302c1f6a2811bb6987c8dd88dc37e9e"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.823302 21552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:33.823938 21578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.824088 21552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:33.824180 21552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "ae1b9f97f045489083a233037d7f03a4"
format_stamp: "Formatted at 2026-08-09 18:55:33 on dist-test-slave-6zbq"
server_key: "7fbbfb91235886931dae6a85bb35b3fe"
server_key_iv: "2302c1f6a2811bb6987c8dd88dc37e9e"
server_key_version: "encryptionkey@0"
I20260809 18:55:33.824246 21552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:33.835635 21552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:33.837116 21552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:33.837244 21552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:33.837431 21552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:33.837688 21552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:33.837721 21552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.837747 21552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:33.837764 21552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:33.842935 21552 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:41269
I20260809 18:55:33.842993 21756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:41269 every 8 connection(s)
I20260809 18:55:33.843262 21552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:55:33.844030 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 21552
I20260809 18:55:33.844131 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:55:33.844316 24427 external_mini_cluster.cc:1428] Setting key 5591d1bb0972acb9378440af911f99d4
Aug 09 18:55:33 dist-test-slave-6zbq krb5kdc[20966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301733, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:33.851621 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.843872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:45885 (local address 127.23.218.254:33107)
0809 18:55:33.843912 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.843914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.844120 (+ 206us) server_negotiation.cc:407] Connection header received
0809 18:55:33.844681 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.844682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.844695 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.844771 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:33.845679 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.846160 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.847131 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.847348 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.847402 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.849363 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:33.849364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:33.849365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:33.849372 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:33.850433 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.850779 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.850780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.850780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.850837 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.851046 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.851047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.851047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.851158 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:33.851214 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.851458 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.851536 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:55:33.851766 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.843851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:33107 (local address 127.23.218.195:45885)
0809 18:55:33.844016 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:33.844053 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:33.844586 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:33.844844 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:33.844847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:33.845422 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:33.845591 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.845596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.846246 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.846247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.847052 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.847054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.847416 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.847418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.847543 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.847961 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:33.847969 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:33.849281 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:33.850507 (+ 1226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:33.850514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:33.850516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:33.850715 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:33.850890 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:33.850891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:33.850891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:33.850990 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:33.851212 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:33.851219 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:33.851344 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 18:55:33.851428 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 18:55:33.851570 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260809 18:55:33.852152 21757 heartbeater.cc:347] Connected to a master server at 127.23.218.254:33107
I20260809 18:55:33.852234 21757 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:33.852363 21757 heartbeater.cc:511] Master 127.23.218.254:33107 requested a full tablet report, sending...
I20260809 18:55:33.852690 21041 ts_manager.cc:194] Registered new tserver with Master: ae1b9f97f045489083a233037d7f03a4 (127.23.218.195:41269)
I20260809 18:55:33.853024 21041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:45885
I20260809 18:55:33.855754 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:55:33.862066 21101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.856923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40040 (local address 127.23.218.254:33107)
0809 18:55:33.856976 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.856977 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.857050 (+ 73us) server_negotiation.cc:407] Connection header received
0809 18:55:33.857109 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.857110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.857118 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.857162 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:33.857666 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.858103 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.858837 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.859015 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.859067 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.860136 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:33.860137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:33.860137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:33.860143 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:33.861168 (+ 1025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.861486 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.861486 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.861486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.861532 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:33.861664 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:33.861665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:33.861665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:33.861777 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:33.861846 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.861933 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.861985 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:55:33.863447 21041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:55:33.864877 21041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:55:33.870893 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.867050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:45091 (local address 127.0.0.1:60948)
0809 18:55:33.867251 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:33.867258 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:33.867291 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:33.867457 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:33.867459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:33.867461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:33.867533 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.867535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.868826 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.868827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.869869 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.869872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.870595 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.870596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.870675 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.870690 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 18:55:33.870762 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 18:55:33.870816 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260809 18:55:33.870929 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.866920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41384 (local address 127.23.218.195:41269)
0809 18:55:33.867575 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.867578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.867586 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:55:33.867629 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.867631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.867644 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.867716 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:33.867849 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.868954 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.869823 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.870626 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.870698 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.870750 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.870822 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.870857 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":68,"threads_started":1}
I20260809 18:55:33.871102 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.866992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51390 (local address 127.23.218.194:33419)
0809 18:55:33.867179 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.867183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.867332 (+ 149us) server_negotiation.cc:407] Connection header received
0809 18:55:33.867371 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.867373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.867387 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.867458 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:33.867606 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.868491 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.869991 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.870681 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.870737 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.870781 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.870975 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.871021 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":78,"threads_started":1}
I20260809 18:55:33.871209 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.867163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.23.218.193:45091)
0809 18:55:33.867310 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.867314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.867324 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:55:33.867358 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.867359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.867371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.867436 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:33.867625 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.868727 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.869958 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.870519 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.870585 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.870640 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.871084 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.871128 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260809 18:55:33.871280 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.866878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:33419 (local address 127.0.0.1:51390)
0809 18:55:33.867224 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:33.867234 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:33.867281 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:33.867471 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:33.867472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:33.867475 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:33.867532 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.867534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.868578 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.868579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.869929 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.869931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.870817 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.870818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.870879 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.870883 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:33.871168 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 18:55:33.871214 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":23,"thread_start_us":84,"threads_started":1}
I20260809 18:55:33.870900 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.866866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41269 (local address 127.0.0.1:41384)
0809 18:55:33.867028 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:33.867067 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:33.867141 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:33.867735 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:33.867736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:33.867738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:33.867778 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.867780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.869036 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.869037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.869746 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.869748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.870696 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.870697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.870761 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.870764 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:33.870812 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 18:55:33.870844 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":45,"threads_started":1}
I20260809 18:55:33.871958 21650 tablet_service.cc:1467] Processing CreateTablet for tablet c64d2d4f01a64377a28c7e9d8abc9ef3 (DEFAULT_TABLE table=test-table [id=f3372ee2824d41b9a47742dc4410a2af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:33.872025 21232 tablet_service.cc:1467] Processing CreateTablet for tablet c64d2d4f01a64377a28c7e9d8abc9ef3 (DEFAULT_TABLE table=test-table [id=f3372ee2824d41b9a47742dc4410a2af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:33.872071 21439 tablet_service.cc:1467] Processing CreateTablet for tablet c64d2d4f01a64377a28c7e9d8abc9ef3 (DEFAULT_TABLE table=test-table [id=f3372ee2824d41b9a47742dc4410a2af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:33.872274 21650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c64d2d4f01a64377a28c7e9d8abc9ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:33.872359 21439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c64d2d4f01a64377a28c7e9d8abc9ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:33.872359 21232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c64d2d4f01a64377a28c7e9d8abc9ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:33.874630 21786 tablet_bootstrap.cc:492] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: Bootstrap starting.
I20260809 18:55:33.875288 21786 tablet_bootstrap.cc:654] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:33.875735 21793 tablet_bootstrap.cc:492] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: Bootstrap starting.
I20260809 18:55:33.875754 21786 log.cc:824] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:33.875830 21789 tablet_bootstrap.cc:492] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b: Bootstrap starting.
I20260809 18:55:33.876399 21786 tablet_bootstrap.cc:492] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: No bootstrap required, opened a new log
I20260809 18:55:33.876451 21789 tablet_bootstrap.cc:654] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:33.876449 21793 tablet_bootstrap.cc:654] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:33.876461 21786 ts_tablet_manager.cc:1397] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:33.877003 21789 log.cc:824] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:33.877003 21793 log.cc:824] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:33.877831 21789 tablet_bootstrap.cc:492] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b: No bootstrap required, opened a new log
I20260809 18:55:33.877902 21789 ts_tablet_manager.cc:1397] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:55:33.878077 21793 tablet_bootstrap.cc:492] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: No bootstrap required, opened a new log
I20260809 18:55:33.878012 21786 raft_consensus.cc:348] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } }
I20260809 18:55:33.878137 21786 raft_consensus.cc:374] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:33.878159 21786 raft_consensus.cc:729] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1b9f97f045489083a233037d7f03a4, State: Initialized, Role: FOLLOWER
I20260809 18:55:33.878206 21793 ts_tablet_manager.cc:1397] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 18:55:33.878304 21786 consensus_queue.cc:260] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } }
I20260809 18:55:33.878590 21786 ts_tablet_manager.cc:1428] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:55:33.878693 21757 heartbeater.cc:503] Master 127.23.218.254:33107 was elected leader, sending a full tablet report...
I20260809 18:55:33.880019 21793 raft_consensus.cc:348] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } }
I20260809 18:55:33.880211 21793 raft_consensus.cc:374] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:33.880260 21793 raft_consensus.cc:729] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 234afbf6adf84d099af841a5a0f4be3a, State: Initialized, Role: FOLLOWER
I20260809 18:55:33.880282 21789 raft_consensus.cc:348] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } }
I20260809 18:55:33.880368 21793 consensus_queue.cc:260] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } }
I20260809 18:55:33.880431 21789 raft_consensus.cc:374] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:33.880473 21789 raft_consensus.cc:729] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606e539c9cd746a0a650a2987b799d1b, State: Initialized, Role: FOLLOWER
I20260809 18:55:33.880621 21793 ts_tablet_manager.cc:1428] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:55:33.880580 21789 consensus_queue.cc:260] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } }
I20260809 18:55:33.880652 21545 heartbeater.cc:503] Master 127.23.218.254:33107 was elected leader, sending a full tablet report...
I20260809 18:55:33.880841 21789 ts_tablet_manager.cc:1428] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 18:55:33.880861 21337 heartbeater.cc:503] Master 127.23.218.254:33107 was elected leader, sending a full tablet report...
W20260809 18:55:33.949229 21546 tablet.cc:2369] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:55:33.976125 21803 raft_consensus.cc:482] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:33.976246 21803 raft_consensus.cc:504] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } }
I20260809 18:55:33.976559 21803 leader_election.cc:288] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae1b9f97f045489083a233037d7f03a4 (127.23.218.195:41269), 234afbf6adf84d099af841a5a0f4be3a (127.23.218.194:33419)
I20260809 18:55:33.980422 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.976885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:33265 (local address 127.23.218.194:33419)
0809 18:55:33.976925 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.976927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.976935 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:55:33.976958 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.976959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.976968 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.977037 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:33.977215 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.977955 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.979466 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.980054 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.980157 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.980166 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.980314 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.980351 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:55:33.980422 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.976810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:46729 (local address 127.23.218.195:41269)
0809 18:55:33.976865 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:33.976867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:33.976979 (+ 112us) server_negotiation.cc:407] Connection header received
0809 18:55:33.977008 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:33.977009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:33.977017 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:33.977067 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:33.977211 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.977957 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.979466 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.980054 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.980132 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.980142 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:33.980314 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 18:55:33.980351 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 18:55:33.980461 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.976720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:33419 (local address 127.23.218.193:33265)
0809 18:55:33.976786 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:33.976798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:33.976840 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:33.977054 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:33.977055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:33.977057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:33.977148 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.977150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.978028 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.978028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.979364 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.979368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.980156 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.980157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.980244 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.980248 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:33.980304 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 18:55:33.980367 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":86}
I20260809 18:55:33.980470 21804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:33.976720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:41269 (local address 127.23.218.193:46729)
0809 18:55:33.976920 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:33.976929 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:33.976970 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:33.977076 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:33.977077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:33.977078 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:33.977148 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.977150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.978028 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.978028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.979367 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:33.979369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:33.980146 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:33.980147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:33.980244 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:33.980249 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:55:33.980304 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 18:55:33.980367 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":70,"thread_start_us":113,"threads_started":1}
I20260809 18:55:33.980844 21474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c64d2d4f01a64377a28c7e9d8abc9ef3" candidate_uuid: "606e539c9cd746a0a650a2987b799d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234afbf6adf84d099af841a5a0f4be3a" is_pre_election: true
I20260809 18:55:33.980858 21685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c64d2d4f01a64377a28c7e9d8abc9ef3" candidate_uuid: "606e539c9cd746a0a650a2987b799d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1b9f97f045489083a233037d7f03a4" is_pre_election: true
I20260809 18:55:33.981000 21685 raft_consensus.cc:2454] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606e539c9cd746a0a650a2987b799d1b in term 0.
I20260809 18:55:33.981000 21474 raft_consensus.cc:2454] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606e539c9cd746a0a650a2987b799d1b in term 0.
I20260809 18:55:33.981248 21162 leader_election.cc:302] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606e539c9cd746a0a650a2987b799d1b, ae1b9f97f045489083a233037d7f03a4; no voters:
I20260809 18:55:33.981402 21803 raft_consensus.cc:2781] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:55:33.981470 21803 raft_consensus.cc:482] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:55:33.981487 21803 raft_consensus.cc:3037] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:33.982199 21803 raft_consensus.cc:504] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } }
I20260809 18:55:33.982493 21474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c64d2d4f01a64377a28c7e9d8abc9ef3" candidate_uuid: "606e539c9cd746a0a650a2987b799d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234afbf6adf84d099af841a5a0f4be3a"
I20260809 18:55:33.982509 21685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c64d2d4f01a64377a28c7e9d8abc9ef3" candidate_uuid: "606e539c9cd746a0a650a2987b799d1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1b9f97f045489083a233037d7f03a4"
I20260809 18:55:33.982568 21474 raft_consensus.cc:3037] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:33.982587 21685 raft_consensus.cc:3037] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:33.983361 21685 raft_consensus.cc:2454] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606e539c9cd746a0a650a2987b799d1b in term 1.
I20260809 18:55:33.983366 21474 raft_consensus.cc:2454] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606e539c9cd746a0a650a2987b799d1b in term 1.
I20260809 18:55:33.983530 21164 leader_election.cc:302] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 234afbf6adf84d099af841a5a0f4be3a, 606e539c9cd746a0a650a2987b799d1b; no voters:
I20260809 18:55:33.984333 21805 raft_consensus.cc:2781] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:33.984336 21803 leader_election.cc:288] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [CANDIDATE]: Term 1 election: Requested vote from peers ae1b9f97f045489083a233037d7f03a4 (127.23.218.195:41269), 234afbf6adf84d099af841a5a0f4be3a (127.23.218.194:33419)
I20260809 18:55:33.984510 21805 raft_consensus.cc:686] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [term 1 LEADER]: Becoming Leader. State: Replica: 606e539c9cd746a0a650a2987b799d1b, State: Running, Role: LEADER
I20260809 18:55:33.984606 21805 consensus_queue.cc:237] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } }
I20260809 18:55:33.985378 21041 catalog_manager.cc:5495] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 606e539c9cd746a0a650a2987b799d1b (127.23.218.193). New cstate: current_term: 1 leader_uuid: "606e539c9cd746a0a650a2987b799d1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "606e539c9cd746a0a650a2987b799d1b" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 45091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 } health_report { overall_health: UNKNOWN } } }
I20260809 18:55:34.016228 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.014135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60950 (local address 127.23.218.193:45091)
0809 18:55:34.014175 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.014177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.014207 (+ 30us) server_negotiation.cc:407] Connection header received
0809 18:55:34.014239 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.014240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.014263 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.014312 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:55:34.014430 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.014871 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.015597 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.015768 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.015810 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.015944 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:55:34.016025 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:55:34.016090 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.016144 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.016156 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:34.017881 21685 raft_consensus.cc:1261] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [term 1 FOLLOWER]: Refusing update from remote peer 606e539c9cd746a0a650a2987b799d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:34.017880 21474 raft_consensus.cc:1261] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [term 1 FOLLOWER]: Refusing update from remote peer 606e539c9cd746a0a650a2987b799d1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:34.018193 21806 consensus_queue.cc:1035] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "234afbf6adf84d099af841a5a0f4be3a" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 33419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:34.018330 21805 consensus_queue.cc:1035] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 606e539c9cd746a0a650a2987b799d1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae1b9f97f045489083a233037d7f03a4" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 41269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:34.021380 21812 mvcc.cc:205] Tried to move back new op lower bound from 7316691902534668288 to 7316691902401466368. Current Snapshot: MvccSnapshot[applied={T|T < 7316691902534668288 or (T in {7316691902534668288})}]
I20260809 18:55:34.023102 21040 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:34.023187 21040 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:34.023891 21040 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60d5cfbc8a1e42e7bcbbda6f92aea671: Sending DeleteTablet for 3 replicas of tablet c64d2d4f01a64377a28c7e9d8abc9ef3
I20260809 18:55:34.024230 21232 tablet_service.cc:1514] Processing DeleteTablet for tablet c64d2d4f01a64377a28c7e9d8abc9ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:34 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:60948
I20260809 18:55:34.024237 21650 tablet_service.cc:1514] Processing DeleteTablet for tablet c64d2d4f01a64377a28c7e9d8abc9ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:34 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:41384
I20260809 18:55:34.024447 21830 tablet_replica.cc:330] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: stopping tablet replica
I20260809 18:55:34.024482 21439 tablet_service.cc:1514] Processing DeleteTablet for tablet c64d2d4f01a64377a28c7e9d8abc9ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:34 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:51390
I20260809 18:55:34.024555 21830 raft_consensus.cc:2229] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:34.024658 21830 raft_consensus.cc:2258] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:34.024854 21832 tablet_replica.cc:330] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: stopping tablet replica
I20260809 18:55:34.024931 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 21135
I20260809 18:55:34.024948 21832 raft_consensus.cc:2229] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:34.025032 21832 raft_consensus.cc:2258] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:34.025718 21832 ts_tablet_manager.cc:1905] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:34.027899 21832 ts_tablet_manager.cc:1918] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:34.027971 21832 log.cc:1196] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/c64d2d4f01a64377a28c7e9d8abc9ef3
I20260809 18:55:34.028221 21832 ts_tablet_manager.cc:1939] T c64d2d4f01a64377a28c7e9d8abc9ef3 P 234afbf6adf84d099af841a5a0f4be3a: Deleting consensus metadata
I20260809 18:55:34.028556 21023 catalog_manager.cc:4841] TS 234afbf6adf84d099af841a5a0f4be3a (127.23.218.194:33419): tablet c64d2d4f01a64377a28c7e9d8abc9ef3 (table test-table [id=f3372ee2824d41b9a47742dc4410a2af]) successfully deleted
I20260809 18:55:34.030472 21830 ts_tablet_manager.cc:1905] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:34.031766 21830 ts_tablet_manager.cc:1918] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:34.031827 21830 log.cc:1196] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/c64d2d4f01a64377a28c7e9d8abc9ef3
I20260809 18:55:34.032044 21830 ts_tablet_manager.cc:1939] T c64d2d4f01a64377a28c7e9d8abc9ef3 P ae1b9f97f045489083a233037d7f03a4: Deleting consensus metadata
I20260809 18:55:34.032426 21018 catalog_manager.cc:4841] TS ae1b9f97f045489083a233037d7f03a4 (127.23.218.195:41269): tablet c64d2d4f01a64377a28c7e9d8abc9ef3 (table test-table [id=f3372ee2824d41b9a47742dc4410a2af]) successfully deleted
W20260809 18:55:34.034812 21018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:45091 (error 108)
I20260809 18:55:34.035050 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 21344
I20260809 18:55:34.037369 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.035208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:45091 (local address 127.0.0.1:60956)
0809 18:55:34.037288 (+ 2080us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.037340 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:45091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2056}
W20260809 18:55:34.037493 21018 catalog_manager.cc:4584] TS 606e539c9cd746a0a650a2987b799d1b (127.23.218.193:45091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c64d2d4f01a64377a28c7e9d8abc9ef3: Network error: Client connection negotiation failed: client connection to 127.23.218.193:45091: connect: Connection refused (error 111)
I20260809 18:55:34.047269 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 21552
I20260809 18:55:34.057801 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 20985
2026-08-09T18:55:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2081 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T18:55:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:34Z Disabled control of system clock
I20260809 18:55:34.090962 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36407
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:36407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:34.169616 21838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:34.169803 21838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:34.169822 21838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:34.170935 21838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:34.170975 21838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:34.170986 21838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:34.171000 21838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:34.171025 21838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:34.172380 21838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:36407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:34.172583 21838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:34.172842 21838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:34.177199 21848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:34.177238 21850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:34.177292 21838 server_base.cc:1034] running on GCE node
W20260809 18:55:34.177199 21854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:34.177637 21838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:34.177866 21838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:34.178993 21838 hybrid_clock.cc:648] HybridClock initialized: now 1786301734178982 us; error 27 us; skew 500 ppm
I20260809 18:55:34.184299 21838 webserver.cc:460] Webserver started at http://127.23.218.254:37479/ using document root <none> and password file <none>
I20260809 18:55:34.184496 21838 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:34.184535 21838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:34.184623 21838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:34.185567 21838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "10b8b6b56b704a7abd482b6b728542b0"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "e1d78fc86953e61f5602069385b3887d"
server_key_iv: "7d9ffe8dc79d0868e7249ec6f800bdee"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.185845 21838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "10b8b6b56b704a7abd482b6b728542b0"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "e1d78fc86953e61f5602069385b3887d"
server_key_iv: "7d9ffe8dc79d0868e7249ec6f800bdee"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.186987 21838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:55:34.187608 21862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.187760 21838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:55:34.187845 21838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "10b8b6b56b704a7abd482b6b728542b0"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "e1d78fc86953e61f5602069385b3887d"
server_key_iv: "7d9ffe8dc79d0868e7249ec6f800bdee"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.187898 21838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:34.197166 21838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:34.197389 21838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:34.197480 21838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:34.201083 21838 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:36407
I20260809 18:55:34.201113 21950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:36407 every 8 connection(s)
I20260809 18:55:34.201397 21838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:34.202234 21951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:34.204273 21951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0: Bootstrap starting.
I20260809 18:55:34.204757 21951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:34.204983 21951 log.cc:824] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:34.205044 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 21838
I20260809 18:55:34.205158 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:34.205359 24427 external_mini_cluster.cc:1428] Setting key cbfda5e24379cc357c282cb9af99a257
I20260809 18:55:34.205736 21951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0: No bootstrap required, opened a new log
I20260809 18:55:34.207554 21951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8b6b56b704a7abd482b6b728542b0" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36407 } }
I20260809 18:55:34.207713 21951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:34.207782 21951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10b8b6b56b704a7abd482b6b728542b0, State: Initialized, Role: FOLLOWER
I20260809 18:55:34.207911 21951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8b6b56b704a7abd482b6b728542b0" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36407 } }
I20260809 18:55:34.207983 21951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:34.208005 21951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:34.208048 21951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:34.208804 21951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8b6b56b704a7abd482b6b728542b0" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36407 } }
I20260809 18:55:34.208885 21951 leader_election.cc:302] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10b8b6b56b704a7abd482b6b728542b0; no voters:
I20260809 18:55:34.209039 21951 leader_election.cc:288] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:34.209085 21959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:34.209228 21959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [term 1 LEADER]: Becoming Leader. State: Replica: 10b8b6b56b704a7abd482b6b728542b0, State: Running, Role: LEADER
I20260809 18:55:34.209293 21951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:34.209357 21959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8b6b56b704a7abd482b6b728542b0" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36407 } }
I20260809 18:55:34.209738 21959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10b8b6b56b704a7abd482b6b728542b0. Latest consensus state: current_term: 1 leader_uuid: "10b8b6b56b704a7abd482b6b728542b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8b6b56b704a7abd482b6b728542b0" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36407 } } }
I20260809 18:55:34.209748 21961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10b8b6b56b704a7abd482b6b728542b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b8b6b56b704a7abd482b6b728542b0" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36407 } } }
I20260809 18:55:34.209908 21959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:34.209921 21961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:34.210171 21974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:34.210696 21974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:34.211927 21958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.205803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.23.218.254:36407)
0809 18:55:34.205988 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.205995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.206047 (+ 52us) server_negotiation.cc:407] Connection header received
0809 18:55:34.207528 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.207537 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.207611 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.207762 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:34.207979 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.208926 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.209926 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.210515 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.210642 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.211255 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:34.211259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:34.211260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:34.211454 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:34.211488 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:34.211499 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:34.211532 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.211644 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.211722 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260809 18:55:34.212599 21974 catalog_manager.cc:1347] Generated new cluster ID: 56184e7673034a0785fc2be71f6540b9
I20260809 18:55:34.212651 21974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:34.228142 21974 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:34.228587 21974 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:34.238386 21974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0: Generated new TSK 0
I20260809 18:55:34.238530 21974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:34.240247 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36407
--builtin_ntp_servers=127.23.218.212:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:34.328686 21988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:34.328871 21988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:34.328890 21988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:34.329990 21988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:34.330027 21988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:34.330050 21988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:55:34.331399 21988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:34.331614 21988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:34.331879 21988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:34.336539 22007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:34.336524 21998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:34.336517 22003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:34.336621 21988 server_base.cc:1034] running on GCE node
I20260809 18:55:34.336953 21988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:34.337183 21988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:34.338371 21988 hybrid_clock.cc:648] HybridClock initialized: now 1786301734338352 us; error 30 us; skew 500 ppm
I20260809 18:55:34.339497 21988 webserver.cc:460] Webserver started at http://127.23.218.193:42055/ using document root <none> and password file <none>
I20260809 18:55:34.339704 21988 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:34.339751 21988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:34.339875 21988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:34.340858 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "4eb5a88cf8d84598b016a0112688ae63"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "ee1f3b7b99adc6068e12b592d18a1b51"
server_key_iv: "51c72546ecd5fede9f96b0d9e1df796e"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.341202 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "4eb5a88cf8d84598b016a0112688ae63"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "ee1f3b7b99adc6068e12b592d18a1b51"
server_key_iv: "51c72546ecd5fede9f96b0d9e1df796e"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.342439 21988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:34.343087 22019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.343250 21988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:34.343330 21988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "4eb5a88cf8d84598b016a0112688ae63"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "ee1f3b7b99adc6068e12b592d18a1b51"
server_key_iv: "51c72546ecd5fede9f96b0d9e1df796e"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.343397 21988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:34.349825 21988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:34.350061 21988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:34.350188 21988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:34.350396 21988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:34.350682 21988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:34.350713 21988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.350739 21988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:34.350765 21988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.356173 21988 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:46457
I20260809 18:55:34.356230 22206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:46457 every 8 connection(s)
I20260809 18:55:34.356487 21988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:55:34.360845 21958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.357084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:59759 (local address 127.23.218.254:36407)
0809 18:55:34.357154 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.357157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.357364 (+ 207us) server_negotiation.cc:407] Connection header received
0809 18:55:34.357815 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.357816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.357827 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.357865 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:34.358475 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.358956 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.359829 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.360000 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.360040 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.360648 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:34.360649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:34.360650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:34.360661 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:34.360676 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:34.360679 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:34.360710 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.360768 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.360791 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 18:55:34.361018 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.357070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36407 (local address 127.23.218.193:59759)
0809 18:55:34.357266 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.357295 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:34.357730 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:34.357913 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:34.357916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:34.357924 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:34.358410 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.358416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.359018 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.359018 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.359771 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.359773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.360049 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.360050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.360148 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.360528 (+ 380us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:55:34.360529 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:34.360554 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:55:34.360574 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:34.360718 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:34.360721 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:34.360723 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:55:34.360765 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 18:55:34.360836 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260809 18:55:34.361451 22207 heartbeater.cc:347] Connected to a master server at 127.23.218.254:36407
I20260809 18:55:34.361531 22207 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:34.361699 22207 heartbeater.cc:511] Master 127.23.218.254:36407 requested a full tablet report, sending...
I20260809 18:55:34.362174 21884 ts_manager.cc:194] Registered new tserver with Master: 4eb5a88cf8d84598b016a0112688ae63 (127.23.218.193:46457)
I20260809 18:55:34.362766 21884 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.193:59759
I20260809 18:55:34.365792 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 21988
I20260809 18:55:34.365893 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:55:34.366067 24427 external_mini_cluster.cc:1428] Setting key c4351151b387ec2ca4389fb8fba0317b
I20260809 18:55:34.366896 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36407
--builtin_ntp_servers=127.23.218.212:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:34.445990 22212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:34.446187 22212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:34.446209 22212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:34.447366 22212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:34.447407 22212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:34.447430 22212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:55:34.448750 22212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:34.448939 22212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:34.449199 22212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:34.453711 22218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:34.453756 22220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:34.453819 22212 server_base.cc:1034] running on GCE node
W20260809 18:55:34.453718 22222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:34.454100 22212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:34.454316 22212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:34.455468 22212 hybrid_clock.cc:648] HybridClock initialized: now 1786301734455455 us; error 28 us; skew 500 ppm
I20260809 18:55:34.456605 22212 webserver.cc:460] Webserver started at http://127.23.218.194:33155/ using document root <none> and password file <none>
I20260809 18:55:34.456835 22212 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:34.456879 22212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:34.457005 22212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:34.457981 22212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "520ff9b168a54edca7d627e4ada24b59"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "1b8974ebd557a7f6bb7f5e67e1d3abc7"
server_key_iv: "c583f20a985e74b63ec35be3d2d07fa0"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.458297 22212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "520ff9b168a54edca7d627e4ada24b59"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "1b8974ebd557a7f6bb7f5e67e1d3abc7"
server_key_iv: "c583f20a985e74b63ec35be3d2d07fa0"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.459453 22212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:55:34.460076 22233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.460209 22212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:34.460290 22212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "520ff9b168a54edca7d627e4ada24b59"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "1b8974ebd557a7f6bb7f5e67e1d3abc7"
server_key_iv: "c583f20a985e74b63ec35be3d2d07fa0"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.460356 22212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:34.481803 22212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:34.482079 22212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:34.482203 22212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:34.482440 22212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:34.482765 22212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:34.482801 22212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.482831 22212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:34.482848 22212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.488612 22212 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:44441
I20260809 18:55:34.488685 22441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:44441 every 8 connection(s)
I20260809 18:55:34.488967 22212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:55:34.491739 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 22212
I20260809 18:55:34.491843 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:55:34.492026 24427 external_mini_cluster.cc:1428] Setting key 31a35ec1ff7d8ddc9155744dcbf981ed
I20260809 18:55:34.492940 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36407
--builtin_ntp_servers=127.23.218.212:34711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 18:55:34.494423 21958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.489649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:39797 (local address 127.23.218.254:36407)
0809 18:55:34.489730 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.489733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.489935 (+ 202us) server_negotiation.cc:407] Connection header received
0809 18:55:34.490508 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.490509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.490522 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.490573 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:34.491243 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.491830 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.493091 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.493299 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.493331 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.494116 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:34.494118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:34.494118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:34.494123 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:34.494143 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:34.494147 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:34.494208 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.494293 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.494324 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 18:55:34.494638 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.489623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36407 (local address 127.23.218.194:39797)
0809 18:55:34.489802 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.489836 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:34.490409 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:34.490641 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:34.490645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:34.490655 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:34.491156 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.491161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.491919 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.491920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.492984 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.492987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.493404 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.493404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.493510 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.493981 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:55:34.493982 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:34.494010 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:55:34.494033 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:34.494214 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:34.494218 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:34.494223 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 18:55:34.494291 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 18:55:34.494433 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260809 18:55:34.495072 22443 heartbeater.cc:347] Connected to a master server at 127.23.218.254:36407
I20260809 18:55:34.495172 22443 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:34.495358 22443 heartbeater.cc:511] Master 127.23.218.254:36407 requested a full tablet report, sending...
I20260809 18:55:34.495735 21884 ts_manager.cc:194] Registered new tserver with Master: 520ff9b168a54edca7d627e4ada24b59 (127.23.218.194:44441)
I20260809 18:55:34.496179 21884 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.194:39797
W20260809 18:55:34.572486 22447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:34.572657 22447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:34.572677 22447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:34.573714 22447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:34.573743 22447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:34.573766 22447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:55:34.575141 22447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:36407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:34.575330 22447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:34.575551 22447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:34.580073 22454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:34.580194 22447 server_base.cc:1034] running on GCE node
W20260809 18:55:34.580169 22457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:34.580142 22453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:34.580447 22447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:34.580668 22447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:34.581807 22447 hybrid_clock.cc:648] HybridClock initialized: now 1786301734581792 us; error 27 us; skew 500 ppm
I20260809 18:55:34.582959 22447 webserver.cc:460] Webserver started at http://127.23.218.195:36763/ using document root <none> and password file <none>
I20260809 18:55:34.583178 22447 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:34.583226 22447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:34.583359 22447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:34.584324 22447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "d84d6e9fffac4817accb35e54b7fa4b1"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "38f22e640f1da891e815603117c8cf7b"
server_key_iv: "985c9aa8975c9a670861d6037b10b9ea"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.584633 22447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "d84d6e9fffac4817accb35e54b7fa4b1"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "38f22e640f1da891e815603117c8cf7b"
server_key_iv: "985c9aa8975c9a670861d6037b10b9ea"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.585829 22447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:55:34.586474 22472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.586620 22447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.586702 22447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "d84d6e9fffac4817accb35e54b7fa4b1"
format_stamp: "Formatted at 2026-08-09 18:55:34 on dist-test-slave-6zbq"
server_key: "38f22e640f1da891e815603117c8cf7b"
server_key_iv: "985c9aa8975c9a670861d6037b10b9ea"
server_key_version: "encryptionkey@0"
I20260809 18:55:34.586769 22447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:34.606202 22447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:34.606494 22447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:34.606611 22447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:34.606786 22447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:34.607143 22447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:34.607191 22447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.607219 22447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:34.607270 22447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:34.613193 22447 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:38677
I20260809 18:55:34.613252 22654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:38677 every 8 connection(s)
I20260809 18:55:34.613516 22447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:55:34.617920 21958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.614126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:50681 (local address 127.23.218.254:36407)
0809 18:55:34.614177 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.614179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.614386 (+ 207us) server_negotiation.cc:407] Connection header received
0809 18:55:34.614848 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.614849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.614858 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.614897 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:34.615478 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.615956 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.616853 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.617044 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.617098 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.617721 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:34.617723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:34.617723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:34.617727 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:34.617741 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:34.617745 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:34.617784 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.617841 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.617864 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 18:55:34.618094 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.614095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36407 (local address 127.23.218.195:50681)
0809 18:55:34.614268 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.614305 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:34.614758 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:34.614947 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:34.614951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:34.614959 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:34.615421 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.615425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.616014 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.616014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.616773 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.616775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.617103 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.617104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.617213 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.617608 (+ 395us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:55:34.617608 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:34.617634 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:55:34.617655 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:34.617791 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:34.617793 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:34.617795 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:55:34.617837 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 18:55:34.617909 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260809 18:55:34.618423 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 22447
I20260809 18:55:34.618528 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:55:34.618537 22657 heartbeater.cc:347] Connected to a master server at 127.23.218.254:36407
I20260809 18:55:34.618618 22657 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:34.618712 24427 external_mini_cluster.cc:1428] Setting key 12d8044e253782bbc23f4a1b3de2e551
I20260809 18:55:34.618750 22657 heartbeater.cc:511] Master 127.23.218.254:36407 requested a full tablet report, sending...
I20260809 18:55:34.619068 21884 ts_manager.cc:194] Registered new tserver with Master: d84d6e9fffac4817accb35e54b7fa4b1 (127.23.218.195:38677)
I20260809 18:55:34.619503 21884 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.195:50681
I20260809 18:55:34.619535 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:55:34.623265 21958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.620717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.23.218.254:36407)
0809 18:55:34.620751 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.620753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.620900 (+ 147us) server_negotiation.cc:407] Connection header received
0809 18:55:34.620952 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.620952 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.620963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.621005 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:34.621182 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.621612 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.622288 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.622515 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.622564 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.623048 (+ 484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:34.623049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:34.623050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:34.623053 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:34.623065 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:34.623068 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:34.623134 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.623179 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.623188 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:55:34.624517 21884 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:55:34.625944 21884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:55:34.631736 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.627951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:38677 (local address 127.0.0.1:40254)
0809 18:55:34.628383 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.628393 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:34.628442 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:34.628641 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:34.628642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:34.628644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:34.628683 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.628685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.629833 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.629834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.630778 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.630780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.631550 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.631550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.631602 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.631612 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 18:55:34.631654 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 18:55:34.631689 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":66,"threads_started":1}
I20260809 18:55:34.631775 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.628155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.23.218.195:38677)
0809 18:55:34.628285 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.628289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.628444 (+ 155us) server_negotiation.cc:407] Connection header received
0809 18:55:34.628497 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.628499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.628514 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.628587 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:34.628753 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.629767 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.630848 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.631473 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.631540 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.631615 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.631668 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.631702 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260809 18:55:34.632280 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.627779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46457 (local address 127.0.0.1:40922)
0809 18:55:34.628069 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.628078 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:34.628132 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:34.628724 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:34.628725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:34.628727 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:34.628779 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.628781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.629631 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.629632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.630960 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.630963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.632064 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.632065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.632113 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.632115 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:55:34.632164 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 18:55:34.632201 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":31,"thread_start_us":85,"threads_started":1}
I20260809 18:55:34.632313 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.627892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.23.218.193:46457)
0809 18:55:34.628563 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.628567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.628576 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:55:34.628614 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.628616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.628631 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.628853 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:34.628909 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.629552 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.631047 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.631772 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.631843 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.631925 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.632194 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.632237 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":81,"threads_started":1}
I20260809 18:55:34.632434 22544 tablet_service.cc:1467] Processing CreateTablet for tablet a0d79798c6ad4451be80c986205add05 (DEFAULT_TABLE table=test-table [id=7cf778b6b3634db3bb7973fcc1e031fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:34.632797 22544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d79798c6ad4451be80c986205add05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:34.633040 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.627764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:44441 (local address 127.0.0.1:51532)
0809 18:55:34.627933 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.627971 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:34.628031 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:34.628312 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:34.628315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:34.628317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:34.628383 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.628385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.629663 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.629663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.630960 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.630963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.632820 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.632821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.632879 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.632882 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:34.632939 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 18:55:34.632975 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":52,"thread_start_us":58,"threads_started":1}
I20260809 18:55:34.633085 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.627918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51532 (local address 127.23.218.194:44441)
0809 18:55:34.628177 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.628181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.628188 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:55:34.628217 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.628219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.628229 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.628295 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:34.628466 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.629591 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.631958 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.632623 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.632703 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.632782 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.632969 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.633011 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":81,"threads_started":1}
I20260809 18:55:34.632994 22095 tablet_service.cc:1467] Processing CreateTablet for tablet a0d79798c6ad4451be80c986205add05 (DEFAULT_TABLE table=test-table [id=7cf778b6b3634db3bb7973fcc1e031fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:34.633314 22095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d79798c6ad4451be80c986205add05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:34.634986 22328 tablet_service.cc:1467] Processing CreateTablet for tablet a0d79798c6ad4451be80c986205add05 (DEFAULT_TABLE table=test-table [id=7cf778b6b3634db3bb7973fcc1e031fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:34.635309 22328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d79798c6ad4451be80c986205add05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:34.635396 22676 tablet_bootstrap.cc:492] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: Bootstrap starting.
I20260809 18:55:34.636024 22676 tablet_bootstrap.cc:654] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:34.636116 22678 tablet_bootstrap.cc:492] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63: Bootstrap starting.
I20260809 18:55:34.636336 22676 log.cc:824] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:34.636715 22678 tablet_bootstrap.cc:654] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:34.636983 22678 log.cc:824] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:34.637112 22676 tablet_bootstrap.cc:492] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: No bootstrap required, opened a new log
I20260809 18:55:34.637167 22676 ts_tablet_manager.cc:1397] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:34.637720 22678 tablet_bootstrap.cc:492] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63: No bootstrap required, opened a new log
I20260809 18:55:34.637779 22678 ts_tablet_manager.cc:1397] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:34.638005 22687 tablet_bootstrap.cc:492] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: Bootstrap starting.
I20260809 18:55:34.638592 22687 tablet_bootstrap.cc:654] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:34.638849 22687 log.cc:824] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:34.639413 22687 tablet_bootstrap.cc:492] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: No bootstrap required, opened a new log
I20260809 18:55:34.639469 22687 ts_tablet_manager.cc:1397] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:34.639647 22676 raft_consensus.cc:348] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } }
I20260809 18:55:34.639822 22676 raft_consensus.cc:374] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:34.639856 22676 raft_consensus.cc:729] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84d6e9fffac4817accb35e54b7fa4b1, State: Initialized, Role: FOLLOWER
I20260809 18:55:34.639999 22676 consensus_queue.cc:260] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } }
I20260809 18:55:34.640241 22676 ts_tablet_manager.cc:1428] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:55:34.640381 22657 heartbeater.cc:503] Master 127.23.218.254:36407 was elected leader, sending a full tablet report...
I20260809 18:55:34.640518 22678 raft_consensus.cc:348] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } }
I20260809 18:55:34.640623 22678 raft_consensus.cc:374] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:34.640641 22678 raft_consensus.cc:729] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eb5a88cf8d84598b016a0112688ae63, State: Initialized, Role: FOLLOWER
I20260809 18:55:34.640723 22678 consensus_queue.cc:260] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } }
I20260809 18:55:34.640944 22678 ts_tablet_manager.cc:1428] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 18:55:34.640990 22207 heartbeater.cc:503] Master 127.23.218.254:36407 was elected leader, sending a full tablet report...
I20260809 18:55:34.641723 22687 raft_consensus.cc:348] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } }
I20260809 18:55:34.641880 22687 raft_consensus.cc:374] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:34.641958 22687 raft_consensus.cc:729] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520ff9b168a54edca7d627e4ada24b59, State: Initialized, Role: FOLLOWER
I20260809 18:55:34.642074 22687 consensus_queue.cc:260] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } }
I20260809 18:55:34.642302 22687 ts_tablet_manager.cc:1428] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:55:34.642378 22443 heartbeater.cc:503] Master 127.23.218.254:36407 was elected leader, sending a full tablet report...
W20260809 18:55:34.739480 22445 tablet.cc:2369] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:55:34.763954 22691 raft_consensus.cc:482] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:34.764063 22691 raft_consensus.cc:504] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } }
I20260809 18:55:34.764436 22691 leader_election.cc:288] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 520ff9b168a54edca7d627e4ada24b59 (127.23.218.194:44441), 4eb5a88cf8d84598b016a0112688ae63 (127.23.218.193:46457)
I20260809 18:55:34.767118 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.764665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:59007 (local address 127.23.218.193:46457)
0809 18:55:34.764708 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.764710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.764834 (+ 124us) server_negotiation.cc:407] Connection header received
0809 18:55:34.764898 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.764899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.764911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.764958 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:34.765104 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.765552 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.766457 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.766813 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.766875 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.766883 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.767016 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.767052 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 18:55:34.767122 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.764640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:38065 (local address 127.23.218.194:44441)
0809 18:55:34.764683 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.764685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.764698 (+ 13us) server_negotiation.cc:407] Connection header received
0809 18:55:34.764796 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.764797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.764807 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.764859 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:34.765001 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.765496 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.766436 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.766804 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.766864 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.766873 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.767017 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.767054 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:34.767161 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.764595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46457 (local address 127.23.218.195:59007)
0809 18:55:34.764763 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.764774 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:34.764837 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:34.764965 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:34.764966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:34.764967 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:34.765044 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.765046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.765622 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.765622 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.766372 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.766375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.766892 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.766893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.766953 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.766956 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:34.767002 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 18:55:34.767042 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260809 18:55:34.767151 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.764585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:44441 (local address 127.23.218.195:38065)
0809 18:55:34.764662 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.764673 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:34.764713 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:34.764871 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:34.764872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:34.764873 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:34.764943 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.764945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.765569 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.765570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.766327 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:34.766329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.766879 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.766880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:34.766952 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.766956 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:34.767001 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 18:55:34.767043 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":13}
I20260809 18:55:34.767598 22130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d79798c6ad4451be80c986205add05" candidate_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eb5a88cf8d84598b016a0112688ae63" is_pre_election: true
I20260809 18:55:34.767635 22365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d79798c6ad4451be80c986205add05" candidate_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "520ff9b168a54edca7d627e4ada24b59" is_pre_election: true
I20260809 18:55:34.767733 22130 raft_consensus.cc:2454] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d84d6e9fffac4817accb35e54b7fa4b1 in term 0.
I20260809 18:55:34.767760 22365 raft_consensus.cc:2454] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d84d6e9fffac4817accb35e54b7fa4b1 in term 0.
I20260809 18:55:34.767953 22475 leader_election.cc:302] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4eb5a88cf8d84598b016a0112688ae63, d84d6e9fffac4817accb35e54b7fa4b1; no voters:
I20260809 18:55:34.768078 22691 raft_consensus.cc:2781] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:55:34.768141 22691 raft_consensus.cc:482] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:55:34.768157 22691 raft_consensus.cc:3037] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:34.768818 22691 raft_consensus.cc:504] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } }
I20260809 18:55:34.768929 22691 leader_election.cc:288] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [CANDIDATE]: Term 1 election: Requested vote from peers 520ff9b168a54edca7d627e4ada24b59 (127.23.218.194:44441), 4eb5a88cf8d84598b016a0112688ae63 (127.23.218.193:46457)
I20260809 18:55:34.769075 22365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d79798c6ad4451be80c986205add05" candidate_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "520ff9b168a54edca7d627e4ada24b59"
I20260809 18:55:34.769142 22130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d79798c6ad4451be80c986205add05" candidate_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eb5a88cf8d84598b016a0112688ae63"
I20260809 18:55:34.769145 22365 raft_consensus.cc:3037] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:34.769220 22130 raft_consensus.cc:3037] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:34.769862 22365 raft_consensus.cc:2454] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d84d6e9fffac4817accb35e54b7fa4b1 in term 1.
I20260809 18:55:34.769894 22130 raft_consensus.cc:2454] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d84d6e9fffac4817accb35e54b7fa4b1 in term 1.
I20260809 18:55:34.770036 22473 leader_election.cc:302] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520ff9b168a54edca7d627e4ada24b59, d84d6e9fffac4817accb35e54b7fa4b1; no voters:
I20260809 18:55:34.770123 22691 raft_consensus.cc:2781] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:34.770249 22691 raft_consensus.cc:686] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 1 LEADER]: Becoming Leader. State: Replica: d84d6e9fffac4817accb35e54b7fa4b1, State: Running, Role: LEADER
I20260809 18:55:34.770332 22691 consensus_queue.cc:237] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } }
I20260809 18:55:34.771085 21883 catalog_manager.cc:5495] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d84d6e9fffac4817accb35e54b7fa4b1 (127.23.218.195). New cstate: current_term: 1 leader_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d84d6e9fffac4817accb35e54b7fa4b1" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 38677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 } health_report { overall_health: UNKNOWN } } }
I20260809 18:55:34.776458 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.774414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.23.218.195:38677)
0809 18:55:34.774450 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:34.774451 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:34.774458 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:55:34.774484 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:34.774485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:34.774501 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:34.774563 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:55:34.774678 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.775094 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.775775 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:34.775958 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:34.776008 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:34.776167 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:55:34.776246 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:55:34.776328 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:34.776372 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 18:55:34.776379 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:34.777876 22365 raft_consensus.cc:1261] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [term 1 FOLLOWER]: Refusing update from remote peer d84d6e9fffac4817accb35e54b7fa4b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:34.777922 22130 raft_consensus.cc:1261] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [term 1 FOLLOWER]: Refusing update from remote peer d84d6e9fffac4817accb35e54b7fa4b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:34.778179 22691 consensus_queue.cc:1035] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "520ff9b168a54edca7d627e4ada24b59" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 44441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:34.778303 22702 consensus_queue.cc:1035] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eb5a88cf8d84598b016a0112688ae63" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 46457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:34.780933 22707 mvcc.cc:205] Tried to move back new op lower bound from 7316691905648295936 to 7316691905619976192. Current Snapshot: MvccSnapshot[applied={T|T < 7316691905648295936}]
I20260809 18:55:34.782619 21884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:34.782696 21884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:34.783346 21884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10b8b6b56b704a7abd482b6b728542b0: Sending DeleteTablet for 3 replicas of tablet a0d79798c6ad4451be80c986205add05
I20260809 18:55:34.783639 22095 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d79798c6ad4451be80c986205add05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:34 UTC) from {username='slave'} at 127.0.0.1:40922
I20260809 18:55:34.783703 22328 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d79798c6ad4451be80c986205add05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:34 UTC) from {username='slave'} at 127.0.0.1:51532
I20260809 18:55:34.783892 22717 tablet_replica.cc:330] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63: stopping tablet replica
I20260809 18:55:34.783978 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 21988
I20260809 18:55:34.783998 22717 raft_consensus.cc:2229] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:34.784082 22717 raft_consensus.cc:2258] T a0d79798c6ad4451be80c986205add05 P 4eb5a88cf8d84598b016a0112688ae63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:34.802032 22544 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d79798c6ad4451be80c986205add05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:34 UTC) from {username='slave'} at 127.0.0.1:40254
I20260809 18:55:34.802271 22719 tablet_replica.cc:330] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: stopping tablet replica
I20260809 18:55:34.802269 22720 tablet_replica.cc:330] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: stopping tablet replica
I20260809 18:55:34.802390 22720 raft_consensus.cc:2229] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:55:34.802464 22719 raft_consensus.cc:2229] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:34.802515 22720 raft_consensus.cc:2258] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:34.802626 22719 raft_consensus.cc:2258] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:34.803050 22720 ts_tablet_manager.cc:1905] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:34.803051 22719 ts_tablet_manager.cc:1905] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:34.804563 22720 ts_tablet_manager.cc:1918] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:34.804598 22719 ts_tablet_manager.cc:1918] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:34.804697 22719 log.cc:1196] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/a0d79798c6ad4451be80c986205add05
I20260809 18:55:34.804706 22720 log.cc:1196] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/a0d79798c6ad4451be80c986205add05
I20260809 18:55:34.804970 22720 ts_tablet_manager.cc:1939] T a0d79798c6ad4451be80c986205add05 P d84d6e9fffac4817accb35e54b7fa4b1: Deleting consensus metadata
I20260809 18:55:34.805037 22719 ts_tablet_manager.cc:1939] T a0d79798c6ad4451be80c986205add05 P 520ff9b168a54edca7d627e4ada24b59: Deleting consensus metadata
I20260809 18:55:34.805394 21863 catalog_manager.cc:4841] TS d84d6e9fffac4817accb35e54b7fa4b1 (127.23.218.195:38677): tablet a0d79798c6ad4451be80c986205add05 (table test-table [id=7cf778b6b3634db3bb7973fcc1e031fd]) successfully deleted
I20260809 18:55:34.805488 21863 catalog_manager.cc:4841] TS 520ff9b168a54edca7d627e4ada24b59 (127.23.218.194:44441): tablet a0d79798c6ad4451be80c986205add05 (table test-table [id=7cf778b6b3634db3bb7973fcc1e031fd]) successfully deleted
W20260809 18:55:34.807155 21865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:46457 (error 108)
I20260809 18:55:34.807282 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 22212
I20260809 18:55:34.814786 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:34.807479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:46457 (local address 127.0.0.1:40936)
0809 18:55:34.814703 (+ 7224us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:34.814754 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7200}
W20260809 18:55:34.835506 21865 catalog_manager.cc:4584] TS 4eb5a88cf8d84598b016a0112688ae63 (127.23.218.193:46457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0d79798c6ad4451be80c986205add05: Network error: Client connection negotiation failed: client connection to 127.23.218.193:46457: connect: Connection refused (error 111)
I20260809 18:55:34.839823 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 22447
I20260809 18:55:34.849308 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 21838
2026-08-09T18:55:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (787 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:34 dist-test-slave-6zbq krb5kdc[22728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:34 dist-test-slave-6zbq krb5kdc[22728](info): set up 2 sockets
Aug 09 18:55:34 dist-test-slave-6zbq krb5kdc[22728](info): commencing operation
krb5kdc: starting...
W20260809 18:55:35.975495 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:36 dist-test-slave-6zbq krb5kdc[22728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:55:36.077862 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34987
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:34987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:36.156153 22745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:36.156337 22745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:36.156365 22745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:36.157505 22745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:36.157544 22745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:36.157563 22745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:36.157580 22745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:36.157598 22745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:36.159122 22745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:34987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:34987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:36.159353 22745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:36.159545 22745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 18:55:36.163960 22745 server_base.cc:1034] running on GCE node
W20260809 18:55:36.163934 22753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:36.163932 22757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:36.163957 22754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:36.164387 22745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:36.164633 22745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:36.165757 22745 hybrid_clock.cc:648] HybridClock initialized: now 1786301736165746 us; error 28 us; skew 500 ppm
Aug 09 18:55:36 dist-test-slave-6zbq krb5kdc[22728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:36.167969 22745 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:55:36.168355 22745 webserver.cc:460] Webserver started at http://127.23.218.254:37883/ using document root <none> and password file <none>
I20260809 18:55:36.168533 22745 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:36.168577 22745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:36.168689 22745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:36.169625 22745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "003410c4c22f4427a8132131ffaf9e0b"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "53bde06d2ceba4feb92554dfc4f67a80"
server_key_iv: "3609aea27ed10d02a071b3da8e088d58"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.169936 22745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "003410c4c22f4427a8132131ffaf9e0b"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "53bde06d2ceba4feb92554dfc4f67a80"
server_key_iv: "3609aea27ed10d02a071b3da8e088d58"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.171147 22745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:36.171772 22767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.171911 22745 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:36.171984 22745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "003410c4c22f4427a8132131ffaf9e0b"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "53bde06d2ceba4feb92554dfc4f67a80"
server_key_iv: "3609aea27ed10d02a071b3da8e088d58"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.172045 22745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:36.178256 22745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:36.179754 22745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:36.179865 22745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:36.183439 22745 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:34987
I20260809 18:55:36.183477 22851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:34987 every 8 connection(s)
I20260809 18:55:36.183743 22745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:36.184480 22853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:36.186532 22853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b: Bootstrap starting.
I20260809 18:55:36.187098 22853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:36.187373 22853 log.cc:824] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:36.187999 22853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b: No bootstrap required, opened a new log
I20260809 18:55:36.189491 22853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003410c4c22f4427a8132131ffaf9e0b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34987 } }
I20260809 18:55:36.189615 22853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:36.189637 22853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003410c4c22f4427a8132131ffaf9e0b, State: Initialized, Role: FOLLOWER
I20260809 18:55:36.189779 22853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003410c4c22f4427a8132131ffaf9e0b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34987 } }
I20260809 18:55:36.189849 22853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:36.189881 22853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:36.189930 22853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:36.190430 22853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003410c4c22f4427a8132131ffaf9e0b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34987 } }
I20260809 18:55:36.190521 22853 leader_election.cc:302] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 003410c4c22f4427a8132131ffaf9e0b; no voters:
I20260809 18:55:36.190670 22853 leader_election.cc:288] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:36.190706 22857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:36.190840 22857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [term 1 LEADER]: Becoming Leader. State: Replica: 003410c4c22f4427a8132131ffaf9e0b, State: Running, Role: LEADER
I20260809 18:55:36.190917 22853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:36.190956 22857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003410c4c22f4427a8132131ffaf9e0b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34987 } }
I20260809 18:55:36.191295 22860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 003410c4c22f4427a8132131ffaf9e0b. Latest consensus state: current_term: 1 leader_uuid: "003410c4c22f4427a8132131ffaf9e0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003410c4c22f4427a8132131ffaf9e0b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34987 } } }
I20260809 18:55:36.191375 22860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:36.191394 22858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "003410c4c22f4427a8132131ffaf9e0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003410c4c22f4427a8132131ffaf9e0b" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 34987 } } }
I20260809 18:55:36.191442 22858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:36.191440 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 22745
I20260809 18:55:36.191551 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:36.191740 24427 external_mini_cluster.cc:1428] Setting key 7997ca4706c18ed4930f7ef5eedc50aa
I20260809 18:55:36.191825 22863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:36.192319 22863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:36.193840 22863 catalog_manager.cc:1347] Generated new cluster ID: c1e62d67e5f04e0db17046e1ce2c321d
I20260809 18:55:36.193892 22863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:55:36 dist-test-slave-6zbq krb5kdc[22728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:36.203554 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.192955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48430 (local address 127.23.218.254:34987)
0809 18:55:36.193095 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.193104 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.193134 (+ 30us) server_negotiation.cc:407] Connection header received
0809 18:55:36.193227 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.193232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.193270 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.193391 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:36.194115 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.195360 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.196363 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.196964 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.197072 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.199621 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:36.199625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:36.199625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:36.199637 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:36.201997 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.202400 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.202403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.202403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.202515 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.202716 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.202717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.202717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.202992 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:36.203115 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.203224 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.203346 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
I20260809 18:55:36.212199 22863 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:36.212648 22863 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:36.221498 22863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b: Generated new TSK 0
I20260809 18:55:36.221635 22863 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.218.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
Entry for principal kudu/127.23.218.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab.
I20260809 18:55:36.252948 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34987
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:36.352228 22896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:36.352404 22896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:36.352425 22896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:36.353468 22896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:36.353500 22896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:36.353524 22896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:55:36.354928 22896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.193.keytab
--principal=kudu/127.23.218.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:36.355103 22896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:36.355268 22896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:36.359999 22909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:36.360054 22908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:36.360006 22911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:36.360256 22896 server_base.cc:1034] running on GCE node
I20260809 18:55:36.360445 22896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:36.360702 22896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:36.361847 22896 hybrid_clock.cc:648] HybridClock initialized: now 1786301736361829 us; error 32 us; skew 500 ppm
Aug 09 18:55:36 dist-test-slave-6zbq krb5kdc[22728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:36.364118 22896 init.cc:377] Logged in from keytab as kudu/127.23.218.193@KRBTEST.COM (short username kudu)
I20260809 18:55:36.364493 22896 webserver.cc:460] Webserver started at http://127.23.218.193:45545/ using document root <none> and password file <none>
I20260809 18:55:36.364681 22896 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:36.364723 22896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:36.364840 22896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:36.365818 22896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "1b229e09f1904e0d91937ad50815b84a"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "fab95d460fb2020602450f82e0ce1c1b"
server_key_iv: "b10bb8f8bab968ba7df12d25ce078c1c"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.366130 22896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "1b229e09f1904e0d91937ad50815b84a"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "fab95d460fb2020602450f82e0ce1c1b"
server_key_iv: "b10bb8f8bab968ba7df12d25ce078c1c"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.367359 22896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 18:55:36.367990 22919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.368135 22896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:36.368222 22896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "1b229e09f1904e0d91937ad50815b84a"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "fab95d460fb2020602450f82e0ce1c1b"
server_key_iv: "b10bb8f8bab968ba7df12d25ce078c1c"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.368289 22896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:36.386960 22896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:36.388827 22896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:36.388947 22896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:36.389133 22896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:36.389400 22896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:36.389430 22896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.389459 22896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:36.389482 22896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.394950 22896 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:39059
I20260809 18:55:36.395009 23112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:39059 every 8 connection(s)
I20260809 18:55:36.395262 22896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:55:36.397661 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 22896
I20260809 18:55:36.397776 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:55:36.397950 24427 external_mini_cluster.cc:1428] Setting key d093776c2598282c286f25a8cae43631
Aug 09 18:55:36 dist-test-slave-6zbq krb5kdc[22728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.193@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:36.404086 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.395826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:43327 (local address 127.23.218.254:34987)
0809 18:55:36.395882 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.395884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.396064 (+ 180us) server_negotiation.cc:407] Connection header received
0809 18:55:36.396515 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.396516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.396531 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.396581 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:36.397430 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.397970 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.399228 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.399427 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.399479 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.401596 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:36.401597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:36.401598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:36.401604 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:36.402798 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.403241 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.403251 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.403252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.403296 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.403540 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.403540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.403540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.403630 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:36.403671 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.403962 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.404025 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 18:55:36.404325 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.395786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34987 (local address 127.23.218.193:43327)
0809 18:55:36.395963 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:36.395996 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:36.396425 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:36.396645 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:36.396649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:36.397169 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:36.397338 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.397343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.398065 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.398067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.399149 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.399153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.399505 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.399506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.399642 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.400031 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:36.400041 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:36.401511 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:36.402880 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:36.402887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:36.402889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:36.403179 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:36.403352 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:36.403352 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:36.403352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:36.403485 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:36.403681 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:36.403687 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:36.403838 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 18:55:36.403952 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 18:55:36.404105 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260809 18:55:36.404786 23114 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34987
I20260809 18:55:36.404870 23114 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:36.405025 23114 heartbeater.cc:511] Master 127.23.218.254:34987 requested a full tablet report, sending...
I20260809 18:55:36.405493 22797 ts_manager.cc:194] Registered new tserver with Master: 1b229e09f1904e0d91937ad50815b84a (127.23.218.193:39059)
I20260809 18:55:36.406049 22797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.193@KRBTEST.COM'} at 127.23.218.193:43327
WARNING: no policy specified for kudu/127.23.218.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
Entry for principal kudu/127.23.218.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab.
I20260809 18:55:36.420306 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34987
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:36.497259 23123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:36.497419 23123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:36.497440 23123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:36.498490 23123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:36.498521 23123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:36.498544 23123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:55:36.499888 23123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.194.keytab
--principal=kudu/127.23.218.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:36.500048 23123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:36.500207 23123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:36.504587 23134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:36.504628 23123 server_base.cc:1034] running on GCE node
W20260809 18:55:36.504596 23131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:36.504601 23137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:36.504937 23123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:36.505131 23123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:36.506261 23123 hybrid_clock.cc:648] HybridClock initialized: now 1786301736506251 us; error 27 us; skew 500 ppm
Aug 09 18:55:36 dist-test-slave-6zbq krb5kdc[22728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:36.508505 23123 init.cc:377] Logged in from keytab as kudu/127.23.218.194@KRBTEST.COM (short username kudu)
I20260809 18:55:36.508898 23123 webserver.cc:460] Webserver started at http://127.23.218.194:43571/ using document root <none> and password file <none>
I20260809 18:55:36.509092 23123 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:36.509136 23123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:36.509248 23123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:36.510135 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "25f457c28aaf4cca951bec54e1a01919"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "f8a0af30fe6e1c56d228b49fd2059df7"
server_key_iv: "b1ff367c8ad798b84a6dbd3e442964a1"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.510459 23123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "25f457c28aaf4cca951bec54e1a01919"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "f8a0af30fe6e1c56d228b49fd2059df7"
server_key_iv: "b1ff367c8ad798b84a6dbd3e442964a1"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.511562 23123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:55:36.512156 23154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.512303 23123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:55:36.512380 23123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "25f457c28aaf4cca951bec54e1a01919"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "f8a0af30fe6e1c56d228b49fd2059df7"
server_key_iv: "b1ff367c8ad798b84a6dbd3e442964a1"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.512439 23123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:36.533747 23123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:36.535632 23123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:36.535755 23123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:36.535940 23123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:36.536198 23123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:36.536230 23123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.536257 23123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:36.536273 23123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.541633 23123 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:45441
I20260809 18:55:36.541692 23341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:45441 every 8 connection(s)
I20260809 18:55:36.541932 23123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:55:36.544235 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 23123
I20260809 18:55:36.544363 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:55:36.544528 24427 external_mini_cluster.cc:1428] Setting key d28a851ad444367cf8029eb5f82fb7dd
Aug 09 18:55:36 dist-test-slave-6zbq krb5kdc[22728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.194@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:36.550541 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.542528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:47535 (local address 127.23.218.254:34987)
0809 18:55:36.542584 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.542586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.542764 (+ 178us) server_negotiation.cc:407] Connection header received
0809 18:55:36.543149 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.543150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.543159 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.543207 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:36.543965 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.544652 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.545690 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.545899 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.545955 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.547972 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:36.547974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:36.547974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:36.547982 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:36.549341 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.549684 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.549685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.549685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.549745 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.549938 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.549939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.549940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.550084 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:36.550159 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.550372 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.550451 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:55:36.550681 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.542510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34987 (local address 127.23.218.194:47535)
0809 18:55:36.542655 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:36.542691 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:36.543070 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:36.543252 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:36.543255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:36.543739 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:36.543902 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.543907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.544736 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.544737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.545601 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.545604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.545968 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.545969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.546098 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.546506 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:36.546514 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:36.547891 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:36.549414 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:36.549420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:36.549422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:36.549617 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:36.549803 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:36.549804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:36.549804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:36.549885 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:36.550146 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:36.550148 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:36.550242 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 18:55:36.550315 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 18:55:36.550485 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260809 18:55:36.551276 23342 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34987
I20260809 18:55:36.551360 23342 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:36.551517 23342 heartbeater.cc:511] Master 127.23.218.254:34987 requested a full tablet report, sending...
I20260809 18:55:36.551868 22797 ts_manager.cc:194] Registered new tserver with Master: 25f457c28aaf4cca951bec54e1a01919 (127.23.218.194:45441)
I20260809 18:55:36.552211 22797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.194@KRBTEST.COM'} at 127.23.218.194:47535
WARNING: no policy specified for kudu/127.23.218.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
Entry for principal kudu/127.23.218.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab.
I20260809 18:55:36.566133 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34987
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:36.647338 23349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:36.647514 23349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:36.647545 23349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:36.648634 23349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:36.648674 23349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:36.648706 23349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:55:36.650115 23349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.195.keytab
--principal=kudu/127.23.218.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:34987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:36.650329 23349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:36.650549 23349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:36.654979 23356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:36.654971 23358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:36.655043 23361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:36.655108 23349 server_base.cc:1034] running on GCE node
I20260809 18:55:36.655400 23349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:36.655639 23349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:36.656771 23349 hybrid_clock.cc:648] HybridClock initialized: now 1786301736656759 us; error 27 us; skew 500 ppm
Aug 09 18:55:36 dist-test-slave-6zbq krb5kdc[22728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:36.658999 23349 init.cc:377] Logged in from keytab as kudu/127.23.218.195@KRBTEST.COM (short username kudu)
I20260809 18:55:36.659384 23349 webserver.cc:460] Webserver started at http://127.23.218.195:33283/ using document root <none> and password file <none>
I20260809 18:55:36.659582 23349 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:36.659636 23349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:36.659755 23349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:36.660674 23349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "55c2c738cdc646c7ac168349148f66b8"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "c8d2ddf99e0ec098a498f0c61087b7f4"
server_key_iv: "7938ffcdb3fcf3097ad8e687b8329b7a"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.660974 23349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "55c2c738cdc646c7ac168349148f66b8"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "c8d2ddf99e0ec098a498f0c61087b7f4"
server_key_iv: "7938ffcdb3fcf3097ad8e687b8329b7a"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.662101 23349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:55:36.662755 23373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.662896 23349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.662976 23349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "55c2c738cdc646c7ac168349148f66b8"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "c8d2ddf99e0ec098a498f0c61087b7f4"
server_key_iv: "7938ffcdb3fcf3097ad8e687b8329b7a"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.663038 23349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:36.670747 23349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:36.672385 23349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:36.672502 23349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:36.672688 23349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:36.672930 23349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:36.672961 23349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.672986 23349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:36.673010 23349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.678668 23349 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:46773
I20260809 18:55:36.678709 23570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:46773 every 8 connection(s)
I20260809 18:55:36.678965 23349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:55:36.679951 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 23349
I20260809 18:55:36.680037 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:55:36.680207 24427 external_mini_cluster.cc:1428] Setting key e2f8f7d3b424eab28eb2daec3aad9dde
Aug 09 18:55:36 dist-test-slave-6zbq krb5kdc[22728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.195@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:36.687155 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.679552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:44693 (local address 127.23.218.254:34987)
0809 18:55:36.679596 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.679598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.679776 (+ 178us) server_negotiation.cc:407] Connection header received
0809 18:55:36.680283 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.680284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.680296 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.680347 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:36.681394 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.681874 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.682851 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.683043 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.683097 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.684924 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:36.684925 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:36.684925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:36.684933 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:36.685945 (+ 1012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.686292 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.686293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.686293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.686370 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.686584 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.686585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.686585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.686691 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:36.686748 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.686992 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.687069 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:36.687286 23575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.679514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:34987 (local address 127.23.218.195:44693)
0809 18:55:36.679673 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:36.679702 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:36.680184 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:36.680422 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:36.680425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:36.681112 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:36.681312 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.681317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.681954 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.681955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.682776 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.682778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.683110 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.683111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.683229 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.683623 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:55:36.683630 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:36.684840 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:36.686016 (+ 1176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:36.686023 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:36.686025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:36.686220 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:36.686425 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:55:36.686426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:55:36.686426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:55:36.686529 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:55:36.686755 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:36.686761 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:36.686878 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 18:55:36.686962 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 18:55:36.687100 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260809 18:55:36.687668 23572 heartbeater.cc:347] Connected to a master server at 127.23.218.254:34987
I20260809 18:55:36.687749 23572 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:36.687872 23572 heartbeater.cc:511] Master 127.23.218.254:34987 requested a full tablet report, sending...
I20260809 18:55:36.688169 22797 ts_manager.cc:194] Registered new tserver with Master: 55c2c738cdc646c7ac168349148f66b8 (127.23.218.195:46773)
I20260809 18:55:36.688486 22797 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.218.195@KRBTEST.COM'} at 127.23.218.195:44693
I20260809 18:55:36.691815 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:55:36.698067 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.693030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.23.218.254:34987)
0809 18:55:36.693063 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.693064 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.693185 (+ 121us) server_negotiation.cc:407] Connection header received
0809 18:55:36.693241 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.693242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.693251 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.693294 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:36.693799 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.694230 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.694945 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.695124 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.695176 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.696170 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:36.696171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:36.696171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:36.696177 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:36.697129 (+ 952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.697409 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.697410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.697410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.697457 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:36.697615 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:36.697616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:36.697616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:36.697734 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:36.697812 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.697920 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.697985 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 18:55:36.699252 22797 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:55:36.700646 22797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:55:36.706090 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.702464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:46773 (local address 127.0.0.1:56502)
0809 18:55:36.702704 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:36.702713 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:36.702753 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:36.702942 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:36.702944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:36.702946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:36.702999 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.703000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.704013 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.704014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.705025 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.705028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.705826 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.705827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.705906 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.705919 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 18:55:36.705974 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 18:55:36.706023 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":49,"threads_started":1}
I20260809 18:55:36.706106 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.702538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56502 (local address 127.23.218.195:46773)
0809 18:55:36.702805 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.702809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.702817 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:55:36.702845 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.702847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.702860 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.702921 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:36.703057 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.703925 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.705109 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.705756 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.705815 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.705865 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.705995 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.706030 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":78,"threads_started":1}
I20260809 18:55:36.706616 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.702461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39059 (local address 127.0.0.1:48750)
0809 18:55:36.702596 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:36.702629 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:36.702686 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:36.703166 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:36.703168 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:36.703170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:36.703222 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.703223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.704636 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.704637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.705522 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.705525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.706387 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.706389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.706454 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.706458 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:36.706522 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 18:55:36.706561 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":46,"threads_started":1}
I20260809 18:55:36.706666 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.702611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.23.218.193:39059)
0809 18:55:36.703008 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.703012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.703021 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:55:36.703058 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.703060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.703073 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.703433 (+ 360us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:36.703515 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.704550 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.705602 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.706286 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.706372 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.706429 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.706539 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.706584 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":70,"threads_started":1}
I20260809 18:55:36.706918 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.702632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:45441 (local address 127.0.0.1:60566)
0809 18:55:36.702760 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:36.702770 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:36.702819 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:36.703100 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:36.703101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:36.703103 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:36.703213 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.703216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.704486 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.704487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.705465 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.705468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.706704 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.706705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.706772 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.706775 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:36.706823 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 18:55:36.706857 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":41,"thread_start_us":42,"threads_started":1}
I20260809 18:55:36.706996 23456 tablet_service.cc:1467] Processing CreateTablet for tablet db2970f9a3c448258cc0e7727e0e89af (DEFAULT_TABLE table=test-table [id=06dff8dc6a934666b2a9b9876c0895c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:36.707149 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.702809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60566 (local address 127.23.218.194:45441)
0809 18:55:36.702959 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.702963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.702973 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:55:36.703009 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.703012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.703027 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.703107 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:36.703295 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.704383 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.705550 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.706274 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.706372 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.706428 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.707030 (+ 602us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.707076 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260809 18:55:36.707320 23456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db2970f9a3c448258cc0e7727e0e89af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:36.707782 23228 tablet_service.cc:1467] Processing CreateTablet for tablet db2970f9a3c448258cc0e7727e0e89af (DEFAULT_TABLE table=test-table [id=06dff8dc6a934666b2a9b9876c0895c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:36.708112 23228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db2970f9a3c448258cc0e7727e0e89af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:36.710417 23596 tablet_bootstrap.cc:492] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: Bootstrap starting.
I20260809 18:55:36.711012 23596 tablet_bootstrap.cc:654] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:36.710906 22999 tablet_service.cc:1467] Processing CreateTablet for tablet db2970f9a3c448258cc0e7727e0e89af (DEFAULT_TABLE table=test-table [id=06dff8dc6a934666b2a9b9876c0895c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:36.711225 22999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db2970f9a3c448258cc0e7727e0e89af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:36.711328 23597 tablet_bootstrap.cc:492] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: Bootstrap starting.
I20260809 18:55:36.711699 23596 log.cc:824] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:36.711941 23597 tablet_bootstrap.cc:654] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:36.712420 23597 log.cc:824] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:36.712450 23596 tablet_bootstrap.cc:492] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: No bootstrap required, opened a new log
I20260809 18:55:36.712502 23596 ts_tablet_manager.cc:1397] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:36.713186 23597 tablet_bootstrap.cc:492] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: No bootstrap required, opened a new log
I20260809 18:55:36.713241 23597 ts_tablet_manager.cc:1397] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:36.713800 23600 tablet_bootstrap.cc:492] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a: Bootstrap starting.
I20260809 18:55:36.714242 23596 raft_consensus.cc:348] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } }
I20260809 18:55:36.714455 23600 tablet_bootstrap.cc:654] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:36.714475 23596 raft_consensus.cc:374] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:36.714546 23596 raft_consensus.cc:729] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55c2c738cdc646c7ac168349148f66b8, State: Initialized, Role: FOLLOWER
I20260809 18:55:36.714674 23596 consensus_queue.cc:260] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } }
I20260809 18:55:36.714973 23596 ts_tablet_manager.cc:1428] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 18:55:36.715011 23572 heartbeater.cc:503] Master 127.23.218.254:34987 was elected leader, sending a full tablet report...
I20260809 18:55:36.715075 23600 log.cc:824] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:36.715540 23597 raft_consensus.cc:348] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } }
I20260809 18:55:36.715684 23597 raft_consensus.cc:374] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:36.715720 23597 raft_consensus.cc:729] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f457c28aaf4cca951bec54e1a01919, State: Initialized, Role: FOLLOWER
I20260809 18:55:36.715828 23597 consensus_queue.cc:260] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } }
I20260809 18:55:36.716082 23597 ts_tablet_manager.cc:1428] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:55:36.716101 23342 heartbeater.cc:503] Master 127.23.218.254:34987 was elected leader, sending a full tablet report...
I20260809 18:55:36.716713 23600 tablet_bootstrap.cc:492] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a: No bootstrap required, opened a new log
I20260809 18:55:36.716784 23600 ts_tablet_manager.cc:1397] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 18:55:36.718113 23600 raft_consensus.cc:348] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } }
I20260809 18:55:36.718261 23600 raft_consensus.cc:374] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:36.718362 23600 raft_consensus.cc:729] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b229e09f1904e0d91937ad50815b84a, State: Initialized, Role: FOLLOWER
I20260809 18:55:36.718487 23600 consensus_queue.cc:260] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } }
I20260809 18:55:36.718726 23600 ts_tablet_manager.cc:1428] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 18:55:36.718770 23114 heartbeater.cc:503] Master 127.23.218.254:34987 was elected leader, sending a full tablet report...
I20260809 18:55:36.754273 23603 raft_consensus.cc:482] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:36.754383 23603 raft_consensus.cc:504] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } }
I20260809 18:55:36.754665 23603 leader_election.cc:288] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55c2c738cdc646c7ac168349148f66b8 (127.23.218.195:46773), 1b229e09f1904e0d91937ad50815b84a (127.23.218.193:39059)
I20260809 18:55:36.757100 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.754806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39059 (local address 127.23.218.194:44615)
0809 18:55:36.754854 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:36.754863 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:36.754898 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:36.755008 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:36.755009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:36.755011 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:36.755075 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.755076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.755665 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.755666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.756361 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.756363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.756877 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.756878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.756950 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.756953 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:36.756998 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 18:55:36.757035 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 18:55:36.757100 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.754861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:44615 (local address 127.23.218.193:39059)
0809 18:55:36.754912 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.754913 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.754920 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:55:36.754941 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.754942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.754951 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.754996 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:36.755133 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.755581 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.756444 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.756805 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.756864 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.756873 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.757012 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.757044 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 18:55:36.757254 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.754809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:46773 (local address 127.23.218.194:57627)
0809 18:55:36.754954 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:36.754962 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:36.755015 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:36.755123 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:36.755124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:36.755125 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:36.755187 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.755188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.755741 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.755749 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.756468 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:36.756471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.757032 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.757033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:36.757095 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.757099 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:36.757143 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 18:55:36.757178 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":11,"thread_start_us":66,"threads_started":1}
I20260809 18:55:36.757238 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.754876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:57627 (local address 127.23.218.195:46773)
0809 18:55:36.754915 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.754917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.755015 (+ 98us) server_negotiation.cc:407] Connection header received
0809 18:55:36.755069 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.755070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.755077 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.755122 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:36.755237 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.755676 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.756542 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.756960 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.757021 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.757029 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.757153 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.757183 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:55:36.757493 23034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db2970f9a3c448258cc0e7727e0e89af" candidate_uuid: "25f457c28aaf4cca951bec54e1a01919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b229e09f1904e0d91937ad50815b84a" is_pre_election: true
I20260809 18:55:36.757619 23034 raft_consensus.cc:2454] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25f457c28aaf4cca951bec54e1a01919 in term 0.
I20260809 18:55:36.757685 23486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db2970f9a3c448258cc0e7727e0e89af" candidate_uuid: "25f457c28aaf4cca951bec54e1a01919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c2c738cdc646c7ac168349148f66b8" is_pre_election: true
I20260809 18:55:36.757824 23486 raft_consensus.cc:2454] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25f457c28aaf4cca951bec54e1a01919 in term 0.
I20260809 18:55:36.757844 23159 leader_election.cc:302] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b229e09f1904e0d91937ad50815b84a, 25f457c28aaf4cca951bec54e1a01919; no voters:
I20260809 18:55:36.757956 23603 raft_consensus.cc:2781] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:55:36.758018 23603 raft_consensus.cc:482] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:55:36.758046 23603 raft_consensus.cc:3037] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:36.758689 23603 raft_consensus.cc:504] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } }
I20260809 18:55:36.758795 23603 leader_election.cc:288] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [CANDIDATE]: Term 1 election: Requested vote from peers 55c2c738cdc646c7ac168349148f66b8 (127.23.218.195:46773), 1b229e09f1904e0d91937ad50815b84a (127.23.218.193:39059)
I20260809 18:55:36.758953 23034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db2970f9a3c448258cc0e7727e0e89af" candidate_uuid: "25f457c28aaf4cca951bec54e1a01919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b229e09f1904e0d91937ad50815b84a"
I20260809 18:55:36.758940 23486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db2970f9a3c448258cc0e7727e0e89af" candidate_uuid: "25f457c28aaf4cca951bec54e1a01919" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55c2c738cdc646c7ac168349148f66b8"
I20260809 18:55:36.759018 23034 raft_consensus.cc:3037] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:36.759018 23486 raft_consensus.cc:3037] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:36.759717 23034 raft_consensus.cc:2454] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25f457c28aaf4cca951bec54e1a01919 in term 1.
I20260809 18:55:36.759742 23486 raft_consensus.cc:2454] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25f457c28aaf4cca951bec54e1a01919 in term 1.
I20260809 18:55:36.759861 23159 leader_election.cc:302] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b229e09f1904e0d91937ad50815b84a, 25f457c28aaf4cca951bec54e1a01919; no voters:
I20260809 18:55:36.759972 23603 raft_consensus.cc:2781] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:36.760105 23603 raft_consensus.cc:686] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 1 LEADER]: Becoming Leader. State: Replica: 25f457c28aaf4cca951bec54e1a01919, State: Running, Role: LEADER
I20260809 18:55:36.760190 23603 consensus_queue.cc:237] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } }
I20260809 18:55:36.760867 22797 catalog_manager.cc:5495] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25f457c28aaf4cca951bec54e1a01919 (127.23.218.194). New cstate: current_term: 1 leader_uuid: "25f457c28aaf4cca951bec54e1a01919" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25f457c28aaf4cca951bec54e1a01919" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 45441 } health_report { overall_health: HEALTHY } } }
I20260809 18:55:36.777369 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.775523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.23.218.194:45441)
0809 18:55:36.775560 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.775562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.775584 (+ 22us) server_negotiation.cc:407] Connection header received
0809 18:55:36.775652 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.775653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.775667 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.775710 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:55:36.775800 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.776204 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.776841 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.777002 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.777041 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.777139 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:55:36.777217 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:55:36.777270 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.777283 (+ 13us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.777296 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:36.778792 23034 raft_consensus.cc:1261] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [term 1 FOLLOWER]: Refusing update from remote peer 25f457c28aaf4cca951bec54e1a01919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:36.778793 23486 raft_consensus.cc:1261] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [term 1 FOLLOWER]: Refusing update from remote peer 25f457c28aaf4cca951bec54e1a01919: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:36.779060 23603 consensus_queue.cc:1035] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55c2c738cdc646c7ac168349148f66b8" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 46773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:36.779181 23607 consensus_queue.cc:1035] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b229e09f1904e0d91937ad50815b84a" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:36.781220 23619 mvcc.cc:205] Tried to move back new op lower bound from 7316691913843793920 to 7316691913770315776. Current Snapshot: MvccSnapshot[applied={T|T < 7316691913843793920}]
I20260809 18:55:36.781359 23616 mvcc.cc:205] Tried to move back new op lower bound from 7316691913843793920 to 7316691913770315776. Current Snapshot: MvccSnapshot[applied={T|T < 7316691913843793920}]
I20260809 18:55:36.783440 22797 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:36.783510 22797 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:36.784097 22797 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 003410c4c22f4427a8132131ffaf9e0b: Sending DeleteTablet for 3 replicas of tablet db2970f9a3c448258cc0e7727e0e89af
I20260809 18:55:36.784417 23456 tablet_service.cc:1514] Processing DeleteTablet for tablet db2970f9a3c448258cc0e7727e0e89af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:36 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:56502
I20260809 18:55:36.784421 22999 tablet_service.cc:1514] Processing DeleteTablet for tablet db2970f9a3c448258cc0e7727e0e89af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:36 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:48750
I20260809 18:55:36.784457 23228 tablet_service.cc:1514] Processing DeleteTablet for tablet db2970f9a3c448258cc0e7727e0e89af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:36 UTC) from {username='kudu', principal='kudu/127.23.218.254@KRBTEST.COM'} at 127.0.0.1:60566
I20260809 18:55:36.784691 23634 tablet_replica.cc:330] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a: stopping tablet replica
I20260809 18:55:36.784695 23632 tablet_replica.cc:330] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: stopping tablet replica
I20260809 18:55:36.784787 23632 raft_consensus.cc:2229] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:55:36.784787 23634 raft_consensus.cc:2229] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:36.784884 23634 raft_consensus.cc:2258] T db2970f9a3c448258cc0e7727e0e89af P 1b229e09f1904e0d91937ad50815b84a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:36.784891 23632 raft_consensus.cc:2258] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:36.784889 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 22896
I20260809 18:55:36.785018 23630 tablet_replica.cc:330] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: stopping tablet replica
I20260809 18:55:36.787359 23630 raft_consensus.cc:2229] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:36.787500 23630 raft_consensus.cc:2258] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:36.789794 23632 ts_tablet_manager.cc:1905] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:36.791092 23632 ts_tablet_manager.cc:1918] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:36.791203 23632 log.cc:1196] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/db2970f9a3c448258cc0e7727e0e89af
I20260809 18:55:36.791525 23632 ts_tablet_manager.cc:1939] T db2970f9a3c448258cc0e7727e0e89af P 25f457c28aaf4cca951bec54e1a01919: Deleting consensus metadata
I20260809 18:55:36.791901 22773 catalog_manager.cc:4841] TS 25f457c28aaf4cca951bec54e1a01919 (127.23.218.194:45441): tablet db2970f9a3c448258cc0e7727e0e89af (table test-table [id=06dff8dc6a934666b2a9b9876c0895c2]) successfully deleted
I20260809 18:55:36.796932 23630 ts_tablet_manager.cc:1905] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:36.798285 23630 ts_tablet_manager.cc:1918] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:36.798377 23630 log.cc:1196] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/db2970f9a3c448258cc0e7727e0e89af
I20260809 18:55:36.798604 23630 ts_tablet_manager.cc:1939] T db2970f9a3c448258cc0e7727e0e89af P 55c2c738cdc646c7ac168349148f66b8: Deleting consensus metadata
I20260809 18:55:36.798967 22773 catalog_manager.cc:4841] TS 55c2c738cdc646c7ac168349148f66b8 (127.23.218.195:46773): tablet db2970f9a3c448258cc0e7727e0e89af (table test-table [id=06dff8dc6a934666b2a9b9876c0895c2]) successfully deleted
W20260809 18:55:36.801152 22778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:39059 (error 108)
I20260809 18:55:36.801429 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 23123
I20260809 18:55:36.801594 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.801459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:39059 (local address 127.0.0.1:48754)
0809 18:55:36.801527 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:36.801566 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:39059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 18:55:36.803378 22778 catalog_manager.cc:4584] TS 1b229e09f1904e0d91937ad50815b84a (127.23.218.193:39059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db2970f9a3c448258cc0e7727e0e89af: Network error: Client connection negotiation failed: client connection to 127.23.218.193:39059: connect: Connection refused (error 111)
I20260809 18:55:36.807060 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 23349
I20260809 18:55:36.813476 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 22745
2026-08-09T18:55:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1964 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T18:55:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:36Z Disabled control of system clock
I20260809 18:55:36.842728 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:45977
--webserver_interface=127.23.218.254
--webserver_port=0
--builtin_ntp_servers=127.23.218.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:45977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:36.919704 23640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:36.919888 23640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:36.919907 23640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:36.920956 23640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:36.920989 23640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:36.921000 23640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:36.921010 23640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:36.921021 23640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:36.922279 23640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:45977
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:45977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:36.922474 23640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:36.922686 23640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:36.926976 23655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:36.927012 23652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:36.927013 23649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:36.927175 23640 server_base.cc:1034] running on GCE node
I20260809 18:55:36.927404 23640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:36.927625 23640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:36.928759 23640 hybrid_clock.cc:648] HybridClock initialized: now 1786301736928748 us; error 29 us; skew 500 ppm
I20260809 18:55:36.929832 23640 webserver.cc:460] Webserver started at http://127.23.218.254:37785/ using document root <none> and password file <none>
I20260809 18:55:36.930032 23640 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:36.930074 23640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:36.930194 23640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:36.931211 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "6be026d275a34fffaa2401d76e922e06"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "d1f05370941a9934fb7e46b7d82bab6c"
server_key_iv: "34f8263901e9643bf6b4d8f76140e277"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.931614 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "6be026d275a34fffaa2401d76e922e06"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "d1f05370941a9934fb7e46b7d82bab6c"
server_key_iv: "34f8263901e9643bf6b4d8f76140e277"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.932791 23640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:55:36.933485 23664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:36.933652 23640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:36.933735 23640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "6be026d275a34fffaa2401d76e922e06"
format_stamp: "Formatted at 2026-08-09 18:55:36 on dist-test-slave-6zbq"
server_key: "d1f05370941a9934fb7e46b7d82bab6c"
server_key_iv: "34f8263901e9643bf6b4d8f76140e277"
server_key_version: "encryptionkey@0"
I20260809 18:55:36.933801 23640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:36.945519 23640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:36.945785 23640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:36.945907 23640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:36.949661 23640 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:45977
I20260809 18:55:36.949698 23760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:45977 every 8 connection(s)
I20260809 18:55:36.949985 23640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:36.950892 23766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:36.953017 23766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06: Bootstrap starting.
I20260809 18:55:36.953531 23766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:36.953761 23766 log.cc:824] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:36.954335 23766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06: No bootstrap required, opened a new log
I20260809 18:55:36.955899 23766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be026d275a34fffaa2401d76e922e06" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 45977 } }
I20260809 18:55:36.956029 23766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:36.956058 23766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6be026d275a34fffaa2401d76e922e06, State: Initialized, Role: FOLLOWER
I20260809 18:55:36.956166 23766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be026d275a34fffaa2401d76e922e06" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 45977 } }
I20260809 18:55:36.956215 23766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:36.956233 23766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:36.956261 23766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:36.956478 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 23640
I20260809 18:55:36.956585 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:36.956756 24427 external_mini_cluster.cc:1428] Setting key fbda795abe30b31ed1546c9df2018146
I20260809 18:55:36.956740 23766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be026d275a34fffaa2401d76e922e06" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 45977 } }
I20260809 18:55:36.956818 23766 leader_election.cc:302] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6be026d275a34fffaa2401d76e922e06; no voters:
I20260809 18:55:36.956967 23766 leader_election.cc:288] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:36.957010 23770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:36.957130 23770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [term 1 LEADER]: Becoming Leader. State: Replica: 6be026d275a34fffaa2401d76e922e06, State: Running, Role: LEADER
I20260809 18:55:36.957284 23770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be026d275a34fffaa2401d76e922e06" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 45977 } }
I20260809 18:55:36.957425 23766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:36.957592 23770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6be026d275a34fffaa2401d76e922e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be026d275a34fffaa2401d76e922e06" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 45977 } } }
I20260809 18:55:36.957739 23770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:36.957906 23770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6be026d275a34fffaa2401d76e922e06. Latest consensus state: current_term: 1 leader_uuid: "6be026d275a34fffaa2401d76e922e06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6be026d275a34fffaa2401d76e922e06" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 45977 } } }
I20260809 18:55:36.957974 23770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:36.958268 23784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:36.958781 23784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:36.960871 23784 catalog_manager.cc:1347] Generated new cluster ID: aeca6bcf07a142549bac6bb406766b7d
I20260809 18:55:36.960925 23784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:36.962424 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:36.957182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.23.218.254:45977)
0809 18:55:36.957363 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:36.957372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:36.957405 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:55:36.957519 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:36.957526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:36.957594 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:36.957975 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:36.958070 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.959475 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.960503 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:36.960941 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:36.961103 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:36.961683 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:36.961687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:36.961687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:36.961854 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:36.961886 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:36.961896 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:36.961953 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:36.962024 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 18:55:36.962184 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260809 18:55:36.981345 23784 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:36.981884 23784 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:36.984617 23784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06: Generated new TSK 0
I20260809 18:55:36.984766 23784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:36.987694 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.193:0
--local_ip_for_outbound_sockets=127.23.218.193
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:45977
--builtin_ntp_servers=127.23.218.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:37.088750 23802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:37.088932 23802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:37.088960 23802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:37.090035 23802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:37.090072 23802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:37.090106 23802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.193
I20260809 18:55:37.091429 23802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.218.193
--webserver_port=0
--tserver_master_addrs=127.23.218.254:45977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.193
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:37.091640 23802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:37.091892 23802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:37.097262 23813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:37.097292 23811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:37.097494 23802 server_base.cc:1034] running on GCE node
W20260809 18:55:37.097262 23816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:37.097769 23802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:37.097982 23802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:37.099115 23802 hybrid_clock.cc:648] HybridClock initialized: now 1786301737099102 us; error 31 us; skew 500 ppm
I20260809 18:55:37.101153 23802 webserver.cc:460] Webserver started at http://127.23.218.193:34915/ using document root <none> and password file <none>
I20260809 18:55:37.101383 23802 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:37.101444 23802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:37.101564 23802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:37.102838 23802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/data/instance:
uuid: "c8419528d780440ab5a0564aa37eaa03"
format_stamp: "Formatted at 2026-08-09 18:55:37 on dist-test-slave-6zbq"
server_key: "a8526a02e3dbd816783f969fc13a9d4e"
server_key_iv: "eba2d7054d8af9431c63449dbf8d6361"
server_key_version: "encryptionkey@0"
I20260809 18:55:37.103104 23802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance:
uuid: "c8419528d780440ab5a0564aa37eaa03"
format_stamp: "Formatted at 2026-08-09 18:55:37 on dist-test-slave-6zbq"
server_key: "a8526a02e3dbd816783f969fc13a9d4e"
server_key_iv: "eba2d7054d8af9431c63449dbf8d6361"
server_key_version: "encryptionkey@0"
I20260809 18:55:37.104200 23802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:55:37.104875 23825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.105050 23802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:37.105132 23802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/wal
uuid: "c8419528d780440ab5a0564aa37eaa03"
format_stamp: "Formatted at 2026-08-09 18:55:37 on dist-test-slave-6zbq"
server_key: "a8526a02e3dbd816783f969fc13a9d4e"
server_key_iv: "eba2d7054d8af9431c63449dbf8d6361"
server_key_version: "encryptionkey@0"
I20260809 18:55:37.105198 23802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:37.121027 23802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:37.121273 23802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:37.121387 23802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:37.121569 23802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:37.121842 23802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:37.121874 23802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.121899 23802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:37.121925 23802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.127391 23802 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.193:40047
I20260809 18:55:37.127429 24017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.193:40047 every 8 connection(s)
I20260809 18:55:37.127709 23802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/data/info.pb
I20260809 18:55:37.132074 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.128329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:43369 (local address 127.23.218.254:45977)
0809 18:55:37.128402 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.128404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.128561 (+ 157us) server_negotiation.cc:407] Connection header received
0809 18:55:37.128994 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.128994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.129005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.129038 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:37.129611 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.130091 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.131022 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.131205 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.131252 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.131882 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:37.131883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:37.131883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:37.131888 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:37.131900 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:37.131904 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:37.131942 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.131997 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.132020 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 18:55:37.132256 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.128305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:45977 (local address 127.23.218.193:43369)
0809 18:55:37.128461 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:37.128491 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:37.128911 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:37.129095 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:37.129098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:37.129106 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:37.129555 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.129560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.130154 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.130154 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.130962 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.130965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.131267 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.131268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.131368 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.131772 (+ 404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:55:37.131772 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:37.131797 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:55:37.131817 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:37.131948 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:37.131950 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:37.131952 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:55:37.131995 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 18:55:37.132068 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260809 18:55:37.132681 24018 heartbeater.cc:347] Connected to a master server at 127.23.218.254:45977
I20260809 18:55:37.132757 24018 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:37.132921 24018 heartbeater.cc:511] Master 127.23.218.254:45977 requested a full tablet report, sending...
I20260809 18:55:37.133359 23697 ts_manager.cc:194] Registered new tserver with Master: c8419528d780440ab5a0564aa37eaa03 (127.23.218.193:40047)
I20260809 18:55:37.133502 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 23802
I20260809 18:55:37.133589 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-0/wal/instance
I20260809 18:55:37.133780 24427 external_mini_cluster.cc:1428] Setting key 82784028c9f1f23c5215bcb5eb10b764
I20260809 18:55:37.134249 23697 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.193:43369
I20260809 18:55:37.134761 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.194:0
--local_ip_for_outbound_sockets=127.23.218.194
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:45977
--builtin_ntp_servers=127.23.218.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:37.213019 24022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:37.213203 24022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:37.213224 24022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:37.214275 24022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:37.214306 24022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:37.214329 24022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.194
I20260809 18:55:37.215674 24022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.218.194
--webserver_port=0
--tserver_master_addrs=127.23.218.254:45977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.194
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:37.215844 24022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:37.216063 24022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:37.220561 24033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:37.220700 24022 server_base.cc:1034] running on GCE node
W20260809 18:55:37.220556 24035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:37.220556 24038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:37.220988 24022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:37.221240 24022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:37.222388 24022 hybrid_clock.cc:648] HybridClock initialized: now 1786301737222371 us; error 27 us; skew 500 ppm
I20260809 18:55:37.223474 24022 webserver.cc:460] Webserver started at http://127.23.218.194:33107/ using document root <none> and password file <none>
I20260809 18:55:37.223676 24022 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:37.223722 24022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:37.223843 24022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:37.224798 24022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/data/instance:
uuid: "919d6b72d9a94397936d1349d72439b3"
format_stamp: "Formatted at 2026-08-09 18:55:37 on dist-test-slave-6zbq"
server_key: "a2bc55d526c8ca2179d0fa549e3eb2d8"
server_key_iv: "7d9b5f450258d3d3b08b9fbc6eeb0021"
server_key_version: "encryptionkey@0"
I20260809 18:55:37.225108 24022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance:
uuid: "919d6b72d9a94397936d1349d72439b3"
format_stamp: "Formatted at 2026-08-09 18:55:37 on dist-test-slave-6zbq"
server_key: "a2bc55d526c8ca2179d0fa549e3eb2d8"
server_key_iv: "7d9b5f450258d3d3b08b9fbc6eeb0021"
server_key_version: "encryptionkey@0"
I20260809 18:55:37.226233 24022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:55:37.226893 24047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.227038 24022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.227136 24022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/wal
uuid: "919d6b72d9a94397936d1349d72439b3"
format_stamp: "Formatted at 2026-08-09 18:55:37 on dist-test-slave-6zbq"
server_key: "a2bc55d526c8ca2179d0fa549e3eb2d8"
server_key_iv: "7d9b5f450258d3d3b08b9fbc6eeb0021"
server_key_version: "encryptionkey@0"
I20260809 18:55:37.227262 24022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:37.255324 24022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:37.255550 24022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:37.255661 24022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:37.255851 24022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:37.256119 24022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:37.256150 24022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.256168 24022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:37.256189 24022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.261534 24022 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.194:34119
I20260809 18:55:37.261602 24213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.194:34119 every 8 connection(s)
I20260809 18:55:37.261849 24022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/data/info.pb
I20260809 18:55:37.266150 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.262467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.194:34497 (local address 127.23.218.254:45977)
0809 18:55:37.262527 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.262529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.262740 (+ 211us) server_negotiation.cc:407] Connection header received
0809 18:55:37.263132 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.263133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.263143 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.263189 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:37.263759 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.264238 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.265105 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.265284 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.265327 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.265975 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:37.265975 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:37.265976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:37.265979 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:37.265990 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:37.265993 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:37.266021 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.266078 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.266096 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:55:37.266327 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.262458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:45977 (local address 127.23.218.194:34497)
0809 18:55:37.262630 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:37.262667 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:37.263061 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:37.263240 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:37.263242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:37.263250 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:37.263693 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.263698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.264301 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.264301 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.265047 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.265049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.265336 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.265336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.265432 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.265868 (+ 436us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:55:37.265868 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:37.265895 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:55:37.265916 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:37.266030 (+ 114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:37.266032 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:37.266034 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:55:37.266073 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 18:55:37.266144 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260809 18:55:37.266733 24216 heartbeater.cc:347] Connected to a master server at 127.23.218.254:45977
I20260809 18:55:37.266809 24216 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:37.266965 24216 heartbeater.cc:511] Master 127.23.218.254:45977 requested a full tablet report, sending...
I20260809 18:55:37.267282 23697 ts_manager.cc:194] Registered new tserver with Master: 919d6b72d9a94397936d1349d72439b3 (127.23.218.194:34119)
I20260809 18:55:37.267625 23697 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.194:34497
I20260809 18:55:37.269709 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 24022
I20260809 18:55:37.269814 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/wal/instance
I20260809 18:55:37.269963 24427 external_mini_cluster.cc:1428] Setting key 88967fff0ce2e00b53fad07eb41498f2
I20260809 18:55:37.270782 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.218.195:0
--local_ip_for_outbound_sockets=127.23.218.195
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:45977
--builtin_ntp_servers=127.23.218.212:42229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 18:55:37.349123 24219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:37.349298 24219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:37.349318 24219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:37.350441 24219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:37.350478 24219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:37.350505 24219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.218.195
I20260809 18:55:37.351821 24219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:42229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.218.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.218.195
--webserver_port=0
--tserver_master_addrs=127.23.218.254:45977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.218.195
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:37.352026 24219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:37.352294 24219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:37.356796 24231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:37.356820 24219 server_base.cc:1034] running on GCE node
W20260809 18:55:37.356796 24228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:37.356823 24229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:37.357180 24219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:37.357407 24219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:37.358551 24219 hybrid_clock.cc:648] HybridClock initialized: now 1786301737358533 us; error 31 us; skew 500 ppm
I20260809 18:55:37.359647 24219 webserver.cc:460] Webserver started at http://127.23.218.195:44349/ using document root <none> and password file <none>
I20260809 18:55:37.359889 24219 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:37.359941 24219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:37.360062 24219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:37.360985 24219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/data/instance:
uuid: "7692690e26404cb588f88da52610dedf"
format_stamp: "Formatted at 2026-08-09 18:55:37 on dist-test-slave-6zbq"
server_key: "1e3a3dacd43425e5825cca8f1402ee7c"
server_key_iv: "30858f6c678699fa92c4dca133b0cb46"
server_key_version: "encryptionkey@0"
I20260809 18:55:37.361316 24219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance:
uuid: "7692690e26404cb588f88da52610dedf"
format_stamp: "Formatted at 2026-08-09 18:55:37 on dist-test-slave-6zbq"
server_key: "1e3a3dacd43425e5825cca8f1402ee7c"
server_key_iv: "30858f6c678699fa92c4dca133b0cb46"
server_key_version: "encryptionkey@0"
I20260809 18:55:37.362457 24219 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:55:37.363078 24243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.363235 24219 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:37.363312 24219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/wal
uuid: "7692690e26404cb588f88da52610dedf"
format_stamp: "Formatted at 2026-08-09 18:55:37 on dist-test-slave-6zbq"
server_key: "1e3a3dacd43425e5825cca8f1402ee7c"
server_key_iv: "30858f6c678699fa92c4dca133b0cb46"
server_key_version: "encryptionkey@0"
I20260809 18:55:37.363377 24219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:37.378901 24219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:37.379120 24219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:37.379240 24219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:37.379413 24219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 18:55:37.379671 24219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 18:55:37.379702 24219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.379727 24219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 18:55:37.379745 24219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:37.384912 24219 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.195:43209
I20260809 18:55:37.384976 24449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.195:43209 every 8 connection(s)
I20260809 18:55:37.385229 24219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/data/info.pb
I20260809 18:55:37.389422 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.385807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.195:54419 (local address 127.23.218.254:45977)
0809 18:55:37.385851 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.385853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.386047 (+ 194us) server_negotiation.cc:407] Connection header received
0809 18:55:37.386484 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.386485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.386494 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.386542 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:37.387096 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.387566 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.388403 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.388573 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.388614 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.389252 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:37.389252 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:37.389253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:37.389256 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:37.389265 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:37.389268 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:37.389296 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.389351 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.389369 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:55:37.389591 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.385769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:45977 (local address 127.23.218.195:54419)
0809 18:55:37.385941 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:37.385976 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:37.386409 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:37.386590 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:37.386593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:37.386600 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:55:37.387037 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.387042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.387625 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.387625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.388347 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.388350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.388624 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.388624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.388726 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.389146 (+ 420us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 18:55:37.389146 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:55:37.389174 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 18:55:37.389194 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:55:37.389305 (+ 111us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:55:37.389307 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:55:37.389309 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 18:55:37.389348 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 18:55:37.389414 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260809 18:55:37.390007 24451 heartbeater.cc:347] Connected to a master server at 127.23.218.254:45977
I20260809 18:55:37.390079 24451 heartbeater.cc:464] Registering TS with master...
I20260809 18:55:37.390213 24451 heartbeater.cc:511] Master 127.23.218.254:45977 requested a full tablet report, sending...
I20260809 18:55:37.390514 23697 ts_manager.cc:194] Registered new tserver with Master: 7692690e26404cb588f88da52610dedf (127.23.218.195:43209)
I20260809 18:55:37.390858 23697 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.218.195:54419
I20260809 18:55:37.395310 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 24219
I20260809 18:55:37.395402 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/wal/instance
I20260809 18:55:37.395525 24427 external_mini_cluster.cc:1428] Setting key 34101786fe1e0fcfa876e0a53e28c456
I20260809 18:55:37.396271 24427 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 18:55:37.399983 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.397464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52804 (local address 127.23.218.254:45977)
0809 18:55:37.397506 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.397507 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.397600 (+ 93us) server_negotiation.cc:407] Connection header received
0809 18:55:37.397661 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.397661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.397674 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.397719 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:37.397864 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.398307 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.399035 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.399217 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.399268 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.399783 (+ 515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:37.399784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:37.399784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:37.399788 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:37.399800 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:37.399804 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:37.399845 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.399893 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.399916 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:55:37.401292 23697 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 18:55:37.402769 23697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 18:55:37.408599 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.404577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:43209 (local address 127.0.0.1:42768)
0809 18:55:37.404925 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:37.404935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:37.404986 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:37.405172 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:37.405173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:37.405174 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:37.405218 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.405219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.406435 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.406436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.407501 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.407504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.408357 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.408358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.408411 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.408420 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 18:55:37.408483 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 18:55:37.408528 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":2,"spinlock_wait_cycles":8960,"thread_start_us":76,"threads_started":1}
I20260809 18:55:37.408612 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.404761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42768 (local address 127.23.218.195:43209)
0809 18:55:37.405029 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.405033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.405041 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:55:37.405071 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.405073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.405086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.405162 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:37.405289 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.406369 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.407577 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.408289 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.408359 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.408437 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.408509 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.408546 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":75,"threads_started":1}
I20260809 18:55:37.408937 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.404566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40047 (local address 127.0.0.1:38920)
0809 18:55:37.404728 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:37.404765 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:37.404826 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:37.405075 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:37.405077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:37.405079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:37.405143 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.405144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.406551 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.406552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.407464 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.407467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.408690 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.408691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.408747 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.408750 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:37.408797 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 18:55:37.408832 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
I20260809 18:55:37.408969 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.404753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38920 (local address 127.23.218.193:40047)
0809 18:55:37.404910 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.404915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.404925 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:55:37.404966 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.404969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.404984 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.405060 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:37.405228 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.406480 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.407548 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.408285 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.408359 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.408437 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.408824 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.408866 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260809 18:55:37.409624 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.404729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55830 (local address 127.23.218.194:34119)
0809 18:55:37.405185 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.405189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.405198 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:55:37.405235 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.405237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.405251 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.405329 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:37.405399 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.406142 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.408476 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.409143 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.409206 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.409296 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.409497 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.409543 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":78,"threads_started":1}
I20260809 18:55:37.409621 24337 tablet_service.cc:1467] Processing CreateTablet for tablet 7d34f7d19fe94731b66ba8f1d372fa79 (DEFAULT_TABLE table=test-table [id=ace1661c147a4b78938d34dfefbdc6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:37.409683 23918 tablet_service.cc:1467] Processing CreateTablet for tablet 7d34f7d19fe94731b66ba8f1d372fa79 (DEFAULT_TABLE table=test-table [id=ace1661c147a4b78938d34dfefbdc6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:37.409852 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.404577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34119 (local address 127.0.0.1:55830)
0809 18:55:37.404906 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:37.404915 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:37.404956 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:37.405326 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:37.405326 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:37.405328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:37.405355 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.405356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.406225 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.406225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.407502 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.407504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.409333 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.409334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.409397 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.409400 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:37.409452 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 18:55:37.409487 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":1,"thread_start_us":73,"threads_started":1}
I20260809 18:55:37.409968 24337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d34f7d19fe94731b66ba8f1d372fa79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:37.410009 23918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d34f7d19fe94731b66ba8f1d372fa79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:37.410567 24123 tablet_service.cc:1467] Processing CreateTablet for tablet 7d34f7d19fe94731b66ba8f1d372fa79 (DEFAULT_TABLE table=test-table [id=ace1661c147a4b78938d34dfefbdc6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 18:55:37.410871 24123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d34f7d19fe94731b66ba8f1d372fa79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:37.412494 24473 tablet_bootstrap.cc:492] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: Bootstrap starting.
I20260809 18:55:37.413172 24474 tablet_bootstrap.cc:492] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03: Bootstrap starting.
I20260809 18:55:37.413213 24473 tablet_bootstrap.cc:654] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:37.413528 24473 log.cc:824] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:37.413894 24474 tablet_bootstrap.cc:654] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:37.414060 24477 tablet_bootstrap.cc:492] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: Bootstrap starting.
I20260809 18:55:37.414259 24473 tablet_bootstrap.cc:492] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: No bootstrap required, opened a new log
I20260809 18:55:37.414242 24474 log.cc:824] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:37.414326 24473 ts_tablet_manager.cc:1397] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:37.414675 24477 tablet_bootstrap.cc:654] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:37.414932 24477 log.cc:824] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:37.415009 24474 tablet_bootstrap.cc:492] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03: No bootstrap required, opened a new log
I20260809 18:55:37.415081 24474 ts_tablet_manager.cc:1397] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 18:55:37.415529 24477 tablet_bootstrap.cc:492] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: No bootstrap required, opened a new log
I20260809 18:55:37.415585 24477 ts_tablet_manager.cc:1397] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 18:55:37.416905 24473 raft_consensus.cc:348] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } }
I20260809 18:55:37.417063 24473 raft_consensus.cc:374] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:37.417097 24473 raft_consensus.cc:729] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7692690e26404cb588f88da52610dedf, State: Initialized, Role: FOLLOWER
I20260809 18:55:37.417253 24473 consensus_queue.cc:260] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } }
I20260809 18:55:37.417510 24474 raft_consensus.cc:348] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } }
I20260809 18:55:37.417620 24473 ts_tablet_manager.cc:1428] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:55:37.417522 24477 raft_consensus.cc:348] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } }
I20260809 18:55:37.417659 24451 heartbeater.cc:503] Master 127.23.218.254:45977 was elected leader, sending a full tablet report...
I20260809 18:55:37.417678 24477 raft_consensus.cc:374] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:37.417711 24477 raft_consensus.cc:729] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919d6b72d9a94397936d1349d72439b3, State: Initialized, Role: FOLLOWER
I20260809 18:55:37.417732 24474 raft_consensus.cc:374] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:37.417798 24474 raft_consensus.cc:729] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8419528d780440ab5a0564aa37eaa03, State: Initialized, Role: FOLLOWER
I20260809 18:55:37.417873 24477 consensus_queue.cc:260] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } }
I20260809 18:55:37.417909 24474 consensus_queue.cc:260] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } }
I20260809 18:55:37.418144 24216 heartbeater.cc:503] Master 127.23.218.254:45977 was elected leader, sending a full tablet report...
I20260809 18:55:37.418159 24474 ts_tablet_manager.cc:1428] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 18:55:37.418397 24018 heartbeater.cc:503] Master 127.23.218.254:45977 was elected leader, sending a full tablet report...
I20260809 18:55:37.418773 24477 ts_tablet_manager.cc:1428] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20260809 18:55:37.512326 24217 tablet.cc:2369] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:55:37.628324 24019 tablet.cc:2369] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 18:55:37.635691 24452 tablet.cc:2369] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:55:37.720882 24488 raft_consensus.cc:482] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 18:55:37.721007 24488 raft_consensus.cc:504] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } }
I20260809 18:55:37.721344 24488 leader_election.cc:288] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 919d6b72d9a94397936d1349d72439b3 (127.23.218.194:34119), 7692690e26404cb588f88da52610dedf (127.23.218.195:43209)
I20260809 18:55:37.724105 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.721587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:40333 (local address 127.23.218.194:34119)
0809 18:55:37.721645 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.721647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.721750 (+ 103us) server_negotiation.cc:407] Connection header received
0809 18:55:37.721814 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.721815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.721825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.721876 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:37.722023 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.722506 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.723443 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.723797 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.723857 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.723866 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.724006 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.724038 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:55:37.724112 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.721595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.218.193:53049 (local address 127.23.218.195:43209)
0809 18:55:37.721664 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.721666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.721796 (+ 130us) server_negotiation.cc:407] Connection header received
0809 18:55:37.721838 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.721839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.721849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.721898 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 18:55:37.722049 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.722551 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.723462 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.723812 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.723871 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.723879 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.724007 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.724046 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 18:55:37.724164 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.721524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.195:43209 (local address 127.23.218.193:53049)
0809 18:55:37.721715 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:37.721726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:37.721788 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:37.721907 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:37.721907 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:37.721909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:37.721984 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.721986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.722622 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.722622 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.723380 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.723382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.723886 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.723887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.723947 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.723951 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 18:55:37.723994 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 18:55:37.724038 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":91,"thread_start_us":59,"threads_started":1}
I20260809 18:55:37.724172 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.721524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.194:34119 (local address 127.23.218.193:40333)
0809 18:55:37.721681 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:37.721690 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:55:37.721755 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:55:37.721890 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:55:37.721891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:55:37.721893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 18:55:37.721967 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.721968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.722578 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.722579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.723365 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:55:37.723367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.723874 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.723875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:55:37.723946 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.723949 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 18:55:37.723995 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 18:55:37.724034 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260809 18:55:37.724502 24373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d34f7d19fe94731b66ba8f1d372fa79" candidate_uuid: "c8419528d780440ab5a0564aa37eaa03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7692690e26404cb588f88da52610dedf" is_pre_election: true
I20260809 18:55:37.724660 24373 raft_consensus.cc:2454] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8419528d780440ab5a0564aa37eaa03 in term 0.
I20260809 18:55:37.724660 24153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d34f7d19fe94731b66ba8f1d372fa79" candidate_uuid: "c8419528d780440ab5a0564aa37eaa03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919d6b72d9a94397936d1349d72439b3" is_pre_election: true
I20260809 18:55:37.724809 24153 raft_consensus.cc:2454] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8419528d780440ab5a0564aa37eaa03 in term 0.
I20260809 18:55:37.724891 23835 leader_election.cc:302] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7692690e26404cb588f88da52610dedf, c8419528d780440ab5a0564aa37eaa03; no voters:
I20260809 18:55:37.725011 24488 raft_consensus.cc:2781] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 18:55:37.725067 24488 raft_consensus.cc:482] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 18:55:37.725098 24488 raft_consensus.cc:3037] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:37.725783 24488 raft_consensus.cc:504] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } }
I20260809 18:55:37.725901 24488 leader_election.cc:288] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [CANDIDATE]: Term 1 election: Requested vote from peers 919d6b72d9a94397936d1349d72439b3 (127.23.218.194:34119), 7692690e26404cb588f88da52610dedf (127.23.218.195:43209)
I20260809 18:55:37.726043 24373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d34f7d19fe94731b66ba8f1d372fa79" candidate_uuid: "c8419528d780440ab5a0564aa37eaa03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7692690e26404cb588f88da52610dedf"
I20260809 18:55:37.726044 24153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d34f7d19fe94731b66ba8f1d372fa79" candidate_uuid: "c8419528d780440ab5a0564aa37eaa03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919d6b72d9a94397936d1349d72439b3"
I20260809 18:55:37.726114 24373 raft_consensus.cc:3037] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:37.726126 24153 raft_consensus.cc:3037] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:37.726876 24373 raft_consensus.cc:2454] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8419528d780440ab5a0564aa37eaa03 in term 1.
I20260809 18:55:37.726948 24153 raft_consensus.cc:2454] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8419528d780440ab5a0564aa37eaa03 in term 1.
I20260809 18:55:37.727022 23835 leader_election.cc:302] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7692690e26404cb588f88da52610dedf, c8419528d780440ab5a0564aa37eaa03; no voters:
I20260809 18:55:37.727104 24488 raft_consensus.cc:2781] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:37.727252 24488 raft_consensus.cc:686] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 1 LEADER]: Becoming Leader. State: Replica: c8419528d780440ab5a0564aa37eaa03, State: Running, Role: LEADER
I20260809 18:55:37.727341 24488 consensus_queue.cc:237] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } }
I20260809 18:55:37.728154 23694 catalog_manager.cc:5495] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8419528d780440ab5a0564aa37eaa03 (127.23.218.193). New cstate: current_term: 1 leader_uuid: "c8419528d780440ab5a0564aa37eaa03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8419528d780440ab5a0564aa37eaa03" member_type: VOTER last_known_addr { host: "127.23.218.193" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 } health_report { overall_health: UNKNOWN } } }
I20260809 18:55:37.756852 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.754755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38924 (local address 127.23.218.193:40047)
0809 18:55:37.754800 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:37.754802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:37.754809 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:55:37.754856 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:37.754857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:37.754876 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:37.754937 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:55:37.755048 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.755477 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.756179 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:37.756378 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:37.756429 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:37.756565 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:55:37.756665 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:55:37.756733 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:37.756773 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 18:55:37.756793 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 18:55:37.758440 24153 raft_consensus.cc:1261] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [term 1 FOLLOWER]: Refusing update from remote peer c8419528d780440ab5a0564aa37eaa03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:37.758450 24373 raft_consensus.cc:1261] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [term 1 FOLLOWER]: Refusing update from remote peer c8419528d780440ab5a0564aa37eaa03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:55:37.758778 24494 consensus_queue.cc:1035] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7692690e26404cb588f88da52610dedf" member_type: VOTER last_known_addr { host: "127.23.218.195" port: 43209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:37.758903 24488 consensus_queue.cc:1035] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919d6b72d9a94397936d1349d72439b3" member_type: VOTER last_known_addr { host: "127.23.218.194" port: 34119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:55:37.761533 24501 mvcc.cc:205] Tried to move back new op lower bound from 7316691917856251904 to 7316691917731749888. Current Snapshot: MvccSnapshot[applied={T|T < 7316691917856251904 or (T in {7316691917856251904})}]
I20260809 18:55:37.761945 24503 mvcc.cc:205] Tried to move back new op lower bound from 7316691917856251904 to 7316691917731749888. Current Snapshot: MvccSnapshot[applied={T|T < 7316691917856251904}]
I20260809 18:55:37.762408 24502 mvcc.cc:205] Tried to move back new op lower bound from 7316691917856251904 to 7316691917731749888. Current Snapshot: MvccSnapshot[applied={T|T < 7316691917856251904}]
I20260809 18:55:37.763224 23694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:37.763309 23694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 18:55:37.763962 23694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6be026d275a34fffaa2401d76e922e06: Sending DeleteTablet for 3 replicas of tablet 7d34f7d19fe94731b66ba8f1d372fa79
I20260809 18:55:37.764277 23918 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d34f7d19fe94731b66ba8f1d372fa79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:37 UTC) from {username='slave'} at 127.0.0.1:38920
I20260809 18:55:37.764310 24337 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d34f7d19fe94731b66ba8f1d372fa79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:37 UTC) from {username='slave'} at 127.0.0.1:42768
I20260809 18:55:37.764351 24123 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d34f7d19fe94731b66ba8f1d372fa79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 18:55:37 UTC) from {username='slave'} at 127.0.0.1:55830
I20260809 18:55:37.764540 24518 tablet_replica.cc:330] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: stopping tablet replica
I20260809 18:55:37.764564 24514 tablet_replica.cc:330] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03: stopping tablet replica
I20260809 18:55:37.764675 24518 raft_consensus.cc:2229] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:37.764689 24514 raft_consensus.cc:2229] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 1 LEADER]: Raft consensus shutting down.
I20260809 18:55:37.764808 24514 raft_consensus.cc:2258] T 7d34f7d19fe94731b66ba8f1d372fa79 P c8419528d780440ab5a0564aa37eaa03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:37.764802 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 23802
I20260809 18:55:37.764806 24518 raft_consensus.cc:2258] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:37.777384 24516 tablet_replica.cc:330] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: stopping tablet replica
I20260809 18:55:37.777514 24516 raft_consensus.cc:2229] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 18:55:37.777592 24518 ts_tablet_manager.cc:1905] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:37.777616 24516 raft_consensus.cc:2258] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 18:55:37.778003 24516 ts_tablet_manager.cc:1905] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 18:55:37.778941 24518 ts_tablet_manager.cc:1918] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:37.779004 24518 log.cc:1196] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-1/wal/wals/7d34f7d19fe94731b66ba8f1d372fa79
I20260809 18:55:37.779217 24518 ts_tablet_manager.cc:1939] T 7d34f7d19fe94731b66ba8f1d372fa79 P 919d6b72d9a94397936d1349d72439b3: Deleting consensus metadata
I20260809 18:55:37.779536 24516 ts_tablet_manager.cc:1918] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 18:55:37.779538 23668 catalog_manager.cc:4841] TS 919d6b72d9a94397936d1349d72439b3 (127.23.218.194:34119): tablet 7d34f7d19fe94731b66ba8f1d372fa79 (table test-table [id=ace1661c147a4b78938d34dfefbdc6dc]) successfully deleted
I20260809 18:55:37.779605 24516 log.cc:1196] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: Deleting WAL directory at /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786301499543878-24427-0/minicluster-data/ts-2/wal/wals/7d34f7d19fe94731b66ba8f1d372fa79
I20260809 18:55:37.779853 24516 ts_tablet_manager.cc:1939] T 7d34f7d19fe94731b66ba8f1d372fa79 P 7692690e26404cb588f88da52610dedf: Deleting consensus metadata
I20260809 18:55:37.780217 23673 catalog_manager.cc:4841] TS 7692690e26404cb588f88da52610dedf (127.23.218.195:43209): tablet 7d34f7d19fe94731b66ba8f1d372fa79 (table test-table [id=ace1661c147a4b78938d34dfefbdc6dc]) successfully deleted
W20260809 18:55:37.782313 23671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.218.193:40047 (error 108)
I20260809 18:55:37.782572 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 24022
I20260809 18:55:37.782722 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:37.782607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.193:40047 (local address 127.0.0.1:38938)
0809 18:55:37.782658 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 18:55:37.782696 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.193:40047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 18:55:37.783942 23671 catalog_manager.cc:4584] TS c8419528d780440ab5a0564aa37eaa03 (127.23.218.193:40047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d34f7d19fe94731b66ba8f1d372fa79: Network error: Client connection negotiation failed: client connection to 127.23.218.193:40047: connect: Connection refused (error 111)
I20260809 18:55:37.790477 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 24219
I20260809 18:55:37.803248 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 23640
2026-08-09T18:55:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (989 ms)
[----------] 6 tests from EncryptionPolicyTest (8901 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:37 dist-test-slave-6zbq krb5kdc[24526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:37 dist-test-slave-6zbq krb5kdc[24526](info): set up 2 sockets
Aug 09 18:55:37 dist-test-slave-6zbq krb5kdc[24526](info): commencing operation
krb5kdc: starting...
W20260809 18:55:38.927438 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:39 dist-test-slave-6zbq krb5kdc[24526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:39.032202 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:39.111334 24550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:39.111509 24550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:39.111529 24550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:39.112627 24550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:39.112666 24550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:39.112679 24550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:39.112703 24550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:39.112725 24550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:39.114143 24550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:39.114387 24550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:39.114583 24550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:39.118935 24561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:39.118997 24550 server_base.cc:1034] running on GCE node
W20260809 18:55:39.118921 24558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:39.118920 24559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:39.119378 24550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:39.119661 24550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:39.120802 24550 hybrid_clock.cc:648] HybridClock initialized: now 1786301739120785 us; error 28 us; skew 500 ppm
Aug 09 18:55:39 dist-test-slave-6zbq krb5kdc[24526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301739, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:39.127496 24550 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:39.127872 24550 webserver.cc:460] Webserver started at http://127.0.0.1:35953/ using document root <none> and password file <none>
I20260809 18:55:39.128072 24550 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:39.128123 24550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:39.128243 24550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:39.129185 24550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "4f94cad4d08c4aeb954d462df35e7c2d"
format_stamp: "Formatted at 2026-08-09 18:55:39 on dist-test-slave-6zbq"
server_key: "b18a913ac7429b1429898a99931d9ea9"
server_key_iv: "4c5a4341a0e56d15c5b0f6f9f1e82084"
server_key_version: "encryptionkey@0"
I20260809 18:55:39.129490 24550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "4f94cad4d08c4aeb954d462df35e7c2d"
format_stamp: "Formatted at 2026-08-09 18:55:39 on dist-test-slave-6zbq"
server_key: "b18a913ac7429b1429898a99931d9ea9"
server_key_iv: "4c5a4341a0e56d15c5b0f6f9f1e82084"
server_key_version: "encryptionkey@0"
I20260809 18:55:39.130779 24550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:39.131414 24571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:39.131556 24550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:39.131642 24550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "4f94cad4d08c4aeb954d462df35e7c2d"
format_stamp: "Formatted at 2026-08-09 18:55:39 on dist-test-slave-6zbq"
server_key: "b18a913ac7429b1429898a99931d9ea9"
server_key_iv: "4c5a4341a0e56d15c5b0f6f9f1e82084"
server_key_version: "encryptionkey@0"
I20260809 18:55:39.131702 24550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:39.139629 24550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:39.141150 24550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:39.141273 24550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:39.144990 24550 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40799
I20260809 18:55:39.145020 24650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40799 every 8 connection(s)
I20260809 18:55:39.145303 24550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:39.146050 24651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:39.146077 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 24550
I20260809 18:55:39.146183 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:39.146386 24427 external_mini_cluster.cc:1428] Setting key 9ba0bb10ed68b13e03a3a0b3b937b483
I20260809 18:55:39.148876 24651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d: Bootstrap starting.
I20260809 18:55:39.149497 24651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:39.149801 24651 log.cc:824] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:39.150470 24651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d: No bootstrap required, opened a new log
Aug 09 18:55:39 dist-test-slave-6zbq krb5kdc[24526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 18:55:39.152750 24651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f94cad4d08c4aeb954d462df35e7c2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40799 } }
I20260809 18:55:39.152875 24651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:39.152907 24651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f94cad4d08c4aeb954d462df35e7c2d, State: Initialized, Role: FOLLOWER
I20260809 18:55:39.153028 24651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f94cad4d08c4aeb954d462df35e7c2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40799 } }
I20260809 18:55:39.153108 24651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:39.153158 24651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:39.153216 24651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:39.153829 24651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f94cad4d08c4aeb954d462df35e7c2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40799 } }
I20260809 18:55:39.153937 24651 leader_election.cc:302] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f94cad4d08c4aeb954d462df35e7c2d; no voters:
I20260809 18:55:39.154089 24651 leader_election.cc:288] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:39.154124 24659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:39.154222 24659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [term 1 LEADER]: Becoming Leader. State: Replica: 4f94cad4d08c4aeb954d462df35e7c2d, State: Running, Role: LEADER
I20260809 18:55:39.154357 24651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:39.154402 24659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f94cad4d08c4aeb954d462df35e7c2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40799 } }
I20260809 18:55:39.154701 24659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f94cad4d08c4aeb954d462df35e7c2d. Latest consensus state: current_term: 1 leader_uuid: "4f94cad4d08c4aeb954d462df35e7c2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f94cad4d08c4aeb954d462df35e7c2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40799 } } }
I20260809 18:55:39.154939 24659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:39.154873 24662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f94cad4d08c4aeb954d462df35e7c2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f94cad4d08c4aeb954d462df35e7c2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40799 } } }
I20260809 18:55:39.155344 24662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:39.155510 24672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:39.156066 24672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:39.156457 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:39.146839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55557 (local address 127.0.0.1:40799)
0809 18:55:39.146989 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:39.146996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:39.147018 (+ 22us) server_negotiation.cc:407] Connection header received
0809 18:55:39.147399 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:39.147405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:39.147447 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:39.147547 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:39.148289 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:39.149163 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:39.150148 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:39.150623 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:39.150705 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:39.153065 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:39.153069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:39.153069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:39.153080 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:39.154791 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:39.155232 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:39.155235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:39.155236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:39.155340 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:39.155590 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:39.155591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:39.155592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:39.155867 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:39.156017 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:39.156180 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 18:55:39.156277 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260809 18:55:39.157331 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:39.157650 24672 catalog_manager.cc:1347] Generated new cluster ID: 0d2f3785f7294f8093009e0c992e3f6b
I20260809 18:55:39.157684 24672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:39.165338 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:39.158815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38375 (local address 127.0.0.1:40799)
0809 18:55:39.158865 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:39.158867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:39.158991 (+ 124us) server_negotiation.cc:407] Connection header received
0809 18:55:39.159044 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:39.159045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:39.159054 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:39.159115 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:39.159645 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:39.160346 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:39.161262 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:39.161574 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:39.161652 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:39.162827 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:39.162828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:39.162829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:39.162835 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:39.164270 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:39.164634 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:39.164635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:39.164635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:39.164693 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:39.164881 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:39.164882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:39.164882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:39.164998 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:39.165086 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:39.165196 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 18:55:39.165257 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:39.174502 24672 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:39.175375 24672 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:39.179193 24672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f94cad4d08c4aeb954d462df35e7c2d: Generated new TSK 0
I20260809 18:55:39.179351 24672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:39.190563 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 24550
W20260809 18:55:39.205159 24524 connection.cc:535] client connection to 127.0.0.1:40799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40799): Connection reset by peer (error 104)
2026-08-09T18:55:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:39 dist-test-slave-6zbq krb5kdc[24700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:39 dist-test-slave-6zbq krb5kdc[24700](info): set up 2 sockets
Aug 09 18:55:39 dist-test-slave-6zbq krb5kdc[24700](info): commencing operation
krb5kdc: starting...
W20260809 18:55:40.331008 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:40 dist-test-slave-6zbq krb5kdc[24700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:40.434427 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:40.512616 24719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:40.512811 24719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:40.512843 24719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:40.513952 24719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:40.513991 24719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:40.514009 24719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:40.514025 24719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:40.514041 24719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:40.515516 24719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41783
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:40.515756 24719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:40.515947 24719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:40.520375 24733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:40.520373 24727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:40.520377 24726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:40.520427 24719 server_base.cc:1034] running on GCE node
I20260809 18:55:40.520779 24719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:40.521030 24719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:40.522186 24719 hybrid_clock.cc:648] HybridClock initialized: now 1786301740522169 us; error 37 us; skew 500 ppm
Aug 09 18:55:40 dist-test-slave-6zbq krb5kdc[24700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301740, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:40.524420 24719 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:40.524814 24719 webserver.cc:460] Webserver started at http://127.0.0.1:43293/ using document root <none> and password file <none>
I20260809 18:55:40.525002 24719 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:40.525039 24719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:40.525125 24719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:40.526021 24719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "ef17cbd669b443efbbd5d632268c1406"
format_stamp: "Formatted at 2026-08-09 18:55:40 on dist-test-slave-6zbq"
server_key: "015d05c7eceeabbbd490a0faab05ab62"
server_key_iv: "99b9265f559e27b274534ae8b1c9ea3e"
server_key_version: "encryptionkey@0"
I20260809 18:55:40.526298 24719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "ef17cbd669b443efbbd5d632268c1406"
format_stamp: "Formatted at 2026-08-09 18:55:40 on dist-test-slave-6zbq"
server_key: "015d05c7eceeabbbd490a0faab05ab62"
server_key_iv: "99b9265f559e27b274534ae8b1c9ea3e"
server_key_version: "encryptionkey@0"
I20260809 18:55:40.527465 24719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:55:40.528103 24746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:40.528258 24719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:40.528328 24719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "ef17cbd669b443efbbd5d632268c1406"
format_stamp: "Formatted at 2026-08-09 18:55:40 on dist-test-slave-6zbq"
server_key: "015d05c7eceeabbbd490a0faab05ab62"
server_key_iv: "99b9265f559e27b274534ae8b1c9ea3e"
server_key_version: "encryptionkey@0"
I20260809 18:55:40.528379 24719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:40.539467 24719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:40.540977 24719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:40.541090 24719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:40.544714 24719 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41783
I20260809 18:55:40.544751 24823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41783 every 8 connection(s)
I20260809 18:55:40.545030 24719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:40.545805 24824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:40.548164 24824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406: Bootstrap starting.
I20260809 18:55:40.548244 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 24719
I20260809 18:55:40.548365 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:40.548519 24427 external_mini_cluster.cc:1428] Setting key 2b772fedc6c48191feba8ad0812f8148
I20260809 18:55:40.548781 24824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:40.549125 24824 log.cc:824] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:40.549854 24824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406: No bootstrap required, opened a new log
I20260809 18:55:40.552179 24824 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17cbd669b443efbbd5d632268c1406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41783 } }
I20260809 18:55:40.552306 24824 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:40.552326 24824 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef17cbd669b443efbbd5d632268c1406, State: Initialized, Role: FOLLOWER
I20260809 18:55:40.552425 24824 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17cbd669b443efbbd5d632268c1406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41783 } }
I20260809 18:55:40.552495 24824 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:40.552516 24824 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:40.552560 24824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:40.553201 24824 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17cbd669b443efbbd5d632268c1406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41783 } }
I20260809 18:55:40.553311 24824 leader_election.cc:302] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef17cbd669b443efbbd5d632268c1406; no voters:
I20260809 18:55:40.553488 24824 leader_election.cc:288] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:40.553503 24831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:40.553646 24831 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [term 1 LEADER]: Becoming Leader. State: Replica: ef17cbd669b443efbbd5d632268c1406, State: Running, Role: LEADER
I20260809 18:55:40.553759 24831 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17cbd669b443efbbd5d632268c1406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41783 } }
I20260809 18:55:40.554023 24824 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:40.554095 24832 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef17cbd669b443efbbd5d632268c1406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17cbd669b443efbbd5d632268c1406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41783 } } }
I20260809 18:55:40.554083 24831 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef17cbd669b443efbbd5d632268c1406. Latest consensus state: current_term: 1 leader_uuid: "ef17cbd669b443efbbd5d632268c1406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef17cbd669b443efbbd5d632268c1406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41783 } } }
I20260809 18:55:40.554206 24832 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:40.554215 24831 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:40.554867 24827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:40.548994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48157 (local address 127.0.0.1:41783)
0809 18:55:40.549171 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:40.549179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:40.549212 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:55:40.549774 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:40.549783 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:40.549840 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:40.549982 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:40.550405 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:40.551781 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:40.552667 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:40.553248 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:40.553354 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:40.554009 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:40.554017 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:40.554018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:40.554216 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:40.554257 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:40.554274 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:40.554381 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:40.554651 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 18:55:40.554681 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260809 18:55:40.555353 24837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:40.555794 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:40.556160 24837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:40.557819 24837 catalog_manager.cc:1347] Generated new cluster ID: 884bf4217b984755b90dc86663d56ec7
I20260809 18:55:40.557871 24837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:40.560559 24827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:40.557397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57065 (local address 127.0.0.1:41783)
0809 18:55:40.557429 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:40.557431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:40.557439 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:55:40.557457 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:40.557457 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:40.557466 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:40.557521 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:40.557660 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:40.558360 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:40.559354 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:40.559675 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:40.559747 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:40.560348 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:40.560349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:40.560349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:40.560354 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:40.560368 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:40.560372 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:40.560432 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:40.560483 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 18:55:40.560492 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:55:40.566885 24837 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:40.567520 24837 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:40.581543 24837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef17cbd669b443efbbd5d632268c1406: Generated new TSK 0
I20260809 18:55:40.581707 24837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:40.589020 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 24719
W20260809 18:55:40.614732 24694 connection.cc:535] client connection to 127.0.0.1:41783 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41783): Connection reset by peer (error 104)
2026-08-09T18:55:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:40 dist-test-slave-6zbq krb5kdc[24870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:40 dist-test-slave-6zbq krb5kdc[24870](info): set up 2 sockets
Aug 09 18:55:40 dist-test-slave-6zbq krb5kdc[24870](info): commencing operation
krb5kdc: starting...
W20260809 18:55:41.737380 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:41 dist-test-slave-6zbq krb5kdc[24870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:41.841445 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:41.919476 24896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:41.919658 24896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:41.919689 24896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:41.920784 24896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:41.920825 24896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:41.920845 24896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:41.920861 24896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:41.920877 24896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:41.922367 24896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46813
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:41.922605 24896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:41.922794 24896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 18:55:41.927186 24896 server_base.cc:1034] running on GCE node
W20260809 18:55:41.927150 24908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:41.927155 24911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:41.927147 24909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:41.927590 24896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:41.927855 24896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:41.928978 24896 hybrid_clock.cc:648] HybridClock initialized: now 1786301741928970 us; error 26 us; skew 500 ppm
Aug 09 18:55:41 dist-test-slave-6zbq krb5kdc[24870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301741, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:41.931303 24896 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:41.931687 24896 webserver.cc:460] Webserver started at http://127.0.0.1:39607/ using document root <none> and password file <none>
I20260809 18:55:41.931890 24896 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:41.931941 24896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:41.932060 24896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:41.933043 24896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "984b6ead4cc64193a28b500bdc125668"
format_stamp: "Formatted at 2026-08-09 18:55:41 on dist-test-slave-6zbq"
server_key: "e6bd0684f97d0a314def5c9cfc3760e9"
server_key_iv: "7e2ef4e25fb030515f478224c359d5fe"
server_key_version: "encryptionkey@0"
I20260809 18:55:41.933360 24896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "984b6ead4cc64193a28b500bdc125668"
format_stamp: "Formatted at 2026-08-09 18:55:41 on dist-test-slave-6zbq"
server_key: "e6bd0684f97d0a314def5c9cfc3760e9"
server_key_iv: "7e2ef4e25fb030515f478224c359d5fe"
server_key_version: "encryptionkey@0"
I20260809 18:55:41.934557 24896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:41.935276 24922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:41.935418 24896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:41.935501 24896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "984b6ead4cc64193a28b500bdc125668"
format_stamp: "Formatted at 2026-08-09 18:55:41 on dist-test-slave-6zbq"
server_key: "e6bd0684f97d0a314def5c9cfc3760e9"
server_key_iv: "7e2ef4e25fb030515f478224c359d5fe"
server_key_version: "encryptionkey@0"
I20260809 18:55:41.935559 24896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:41.938642 24896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:41.940310 24896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:41.940428 24896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:41.944125 24896 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46813
I20260809 18:55:41.944170 25002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46813 every 8 connection(s)
I20260809 18:55:41.944449 24896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:41.945129 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 24896
I20260809 18:55:41.945254 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:41.945441 24427 external_mini_cluster.cc:1428] Setting key cc972caed357201b67c576b6d61d4ac3
I20260809 18:55:41.945510 25003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:41.947800 25003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668: Bootstrap starting.
I20260809 18:55:41.948313 25003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:41.948549 25003 log.cc:824] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:41.949152 25003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668: No bootstrap required, opened a new log
I20260809 18:55:41.950503 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:41.945885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56251 (local address 127.0.0.1:46813)
0809 18:55:41.946064 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:41.946073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:41.948860 (+ 2787us) server_negotiation.cc:407] Connection header received
0809 18:55:41.949311 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:41.949318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:41.949343 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:41.949463 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:41.950088 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:41.950092 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:41.950093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:41.950097 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:41.950137 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:41.950147 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:41.950220 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:41.950286 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 18:55:41.950325 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260809 18:55:41.951375 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:41.951290 25003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984b6ead4cc64193a28b500bdc125668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } }
I20260809 18:55:41.951437 25003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:41.951483 25003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984b6ead4cc64193a28b500bdc125668, State: Initialized, Role: FOLLOWER
I20260809 18:55:41.951601 25003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984b6ead4cc64193a28b500bdc125668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } }
I20260809 18:55:41.951689 25003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:41.951720 25003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:41.951769 25003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:41.952255 25003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984b6ead4cc64193a28b500bdc125668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } }
I20260809 18:55:41.952353 25003 leader_election.cc:302] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 984b6ead4cc64193a28b500bdc125668; no voters:
I20260809 18:55:41.952502 25003 leader_election.cc:288] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:41.952541 25016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:41.952679 25016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [term 1 LEADER]: Becoming Leader. State: Replica: 984b6ead4cc64193a28b500bdc125668, State: Running, Role: LEADER
I20260809 18:55:41.952782 25003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:41.952812 25016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984b6ead4cc64193a28b500bdc125668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } }
I20260809 18:55:41.953123 25016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 984b6ead4cc64193a28b500bdc125668. Latest consensus state: current_term: 1 leader_uuid: "984b6ead4cc64193a28b500bdc125668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984b6ead4cc64193a28b500bdc125668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } } }
I20260809 18:55:41.953197 25018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "984b6ead4cc64193a28b500bdc125668" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984b6ead4cc64193a28b500bdc125668" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } } }
I20260809 18:55:41.953258 25016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:41.953697 25018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:41.953902 25039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:41.954384 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:41.952717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51891 (local address 127.0.0.1:46813)
0809 18:55:41.952771 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:41.952772 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:41.952994 (+ 222us) server_negotiation.cc:407] Connection header received
0809 18:55:41.953062 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:41.953063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:41.953068 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:41.953121 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:41.954076 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:41.954077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:41.954077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:41.954082 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:41.954095 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:41.954099 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:41.954144 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:41.954296 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 18:55:41.954309 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:55:41.954424 25039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:41.955969 25039 catalog_manager.cc:1347] Generated new cluster ID: 2b89b0a6266848308deab241ae26be67
I20260809 18:55:41.956024 25039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:41.972739 25039 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:41.972896 25039 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:41.982721 25039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 984b6ead4cc64193a28b500bdc125668: Generated new TSK 0
I20260809 18:55:41.982892 25039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:41.986075 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 24896
2026-08-09T18:55:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:42 dist-test-slave-6zbq krb5kdc[25058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:42 dist-test-slave-6zbq krb5kdc[25058](info): set up 2 sockets
Aug 09 18:55:42 dist-test-slave-6zbq krb5kdc[25058](info): commencing operation
krb5kdc: starting...
W20260809 18:55:43.131102 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:43 dist-test-slave-6zbq krb5kdc[25058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:43.233978 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46855
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:43.312220 25083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:43.312408 25083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:43.312438 25083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:43.313565 25083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:43.313606 25083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:43.313625 25083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:43.313642 25083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:43.313657 25083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:43.315100 25083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:43.315326 25083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:43.315519 25083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:43.320025 25100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:43.320004 25094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:43.320034 25083 server_base.cc:1034] running on GCE node
W20260809 18:55:43.319996 25095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:43.320391 25083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:43.320652 25083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:43.321796 25083 hybrid_clock.cc:648] HybridClock initialized: now 1786301743321786 us; error 29 us; skew 500 ppm
Aug 09 18:55:43 dist-test-slave-6zbq krb5kdc[25058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301743, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:43.323974 25083 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:43.324370 25083 webserver.cc:460] Webserver started at http://127.0.0.1:42347/ using document root <none> and password file <none>
I20260809 18:55:43.324553 25083 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:43.324599 25083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:43.324724 25083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:43.325657 25083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "037024a6cf3e4ae6943e2f517bb3138e"
format_stamp: "Formatted at 2026-08-09 18:55:43 on dist-test-slave-6zbq"
server_key: "2bce804969b20d9aea158a2b27d2aff6"
server_key_iv: "c2c753da4bac942b5c8bd598b454e068"
server_key_version: "encryptionkey@0"
I20260809 18:55:43.325981 25083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "037024a6cf3e4ae6943e2f517bb3138e"
format_stamp: "Formatted at 2026-08-09 18:55:43 on dist-test-slave-6zbq"
server_key: "2bce804969b20d9aea158a2b27d2aff6"
server_key_iv: "c2c753da4bac942b5c8bd598b454e068"
server_key_version: "encryptionkey@0"
I20260809 18:55:43.327211 25083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:43.327847 25111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:43.327987 25083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:43.328068 25083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "037024a6cf3e4ae6943e2f517bb3138e"
format_stamp: "Formatted at 2026-08-09 18:55:43 on dist-test-slave-6zbq"
server_key: "2bce804969b20d9aea158a2b27d2aff6"
server_key_iv: "c2c753da4bac942b5c8bd598b454e068"
server_key_version: "encryptionkey@0"
I20260809 18:55:43.328125 25083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:43.351125 25083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:43.352705 25083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:43.352828 25083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:43.356490 25083 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46855
I20260809 18:55:43.356519 25190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46855 every 8 connection(s)
I20260809 18:55:43.356815 25083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:43.357602 25191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:43.357785 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25083
I20260809 18:55:43.357903 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:43.358095 24427 external_mini_cluster.cc:1428] Setting key 01e4aa63439827b0c03fa0010df885dc
I20260809 18:55:43.360345 25191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e: Bootstrap starting.
I20260809 18:55:43.360889 25191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:43.361191 25191 log.cc:824] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:43.361860 25191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e: No bootstrap required, opened a new log
I20260809 18:55:43.363785 25191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037024a6cf3e4ae6943e2f517bb3138e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } }
I20260809 18:55:43.363948 25191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:43.363993 25191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 037024a6cf3e4ae6943e2f517bb3138e, State: Initialized, Role: FOLLOWER
I20260809 18:55:43.364116 25191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037024a6cf3e4ae6943e2f517bb3138e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } }
I20260809 18:55:43.364192 25191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:43.364226 25191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 09 18:55:43 dist-test-slave-6zbq krb5kdc[25058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 18:55:43.364281 25191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:43.364774 25191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037024a6cf3e4ae6943e2f517bb3138e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } }
I20260809 18:55:43.364862 25191 leader_election.cc:302] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 037024a6cf3e4ae6943e2f517bb3138e; no voters:
I20260809 18:55:43.365010 25191 leader_election.cc:288] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:43.365056 25197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:43.365212 25197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [term 1 LEADER]: Becoming Leader. State: Replica: 037024a6cf3e4ae6943e2f517bb3138e, State: Running, Role: LEADER
I20260809 18:55:43.365312 25191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:43.365370 25197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037024a6cf3e4ae6943e2f517bb3138e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } }
I20260809 18:55:43.365711 25201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "037024a6cf3e4ae6943e2f517bb3138e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037024a6cf3e4ae6943e2f517bb3138e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } } }
I20260809 18:55:43.365805 25201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:43.365721 25202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 037024a6cf3e4ae6943e2f517bb3138e. Latest consensus state: current_term: 1 leader_uuid: "037024a6cf3e4ae6943e2f517bb3138e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "037024a6cf3e4ae6943e2f517bb3138e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } } }
I20260809 18:55:43.365918 25202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:43.366068 25213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:43.366801 25213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:43.368441 25213 catalog_manager.cc:1347] Generated new cluster ID: 29c180cc94de47a3acae2ad9150dd241
I20260809 18:55:43.368454 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:43.358569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39055 (local address 127.0.0.1:46855)
0809 18:55:43.358741 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:43.358750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:43.358776 (+ 26us) server_negotiation.cc:407] Connection header received
0809 18:55:43.359185 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:43.359191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:43.359235 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:43.359340 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:43.360076 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:43.360950 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:43.361901 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:43.362359 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:43.362452 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:43.364673 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:43.364678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:43.364678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:43.364689 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:43.366880 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:43.367273 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:43.367276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:43.367276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:43.367379 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:43.367575 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:43.367576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:43.367576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:43.367857 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:43.367997 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:43.368129 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 18:55:43.368263 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260809 18:55:43.368494 25213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:43.369341 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:43.380817 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:43.374425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38959 (local address 127.0.0.1:46855)
0809 18:55:43.374561 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:43.374562 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:43.374569 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:55:43.374586 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:43.374587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:43.374595 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:43.374645 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:43.375231 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:43.375925 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:43.376913 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:43.377210 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:43.377262 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:43.378461 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:43.378463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:43.378463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:43.378469 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:43.379824 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:43.380170 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:43.380170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:43.380171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:43.380216 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:43.380384 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:43.380385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:43.380385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:43.380484 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:43.380553 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:43.380668 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 18:55:43.380737 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":121}
I20260809 18:55:43.391108 25213 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:43.391548 25213 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:43.394701 25213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 037024a6cf3e4ae6943e2f517bb3138e: Generated new TSK 0
I20260809 18:55:43.394843 25213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:43.400223 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25083
2026-08-09T18:55:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:43 dist-test-slave-6zbq krb5kdc[25240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:43 dist-test-slave-6zbq krb5kdc[25240](info): set up 2 sockets
Aug 09 18:55:43 dist-test-slave-6zbq krb5kdc[25240](info): commencing operation
krb5kdc: starting...
W20260809 18:55:44.552726 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:44 dist-test-slave-6zbq krb5kdc[25240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:44.661509 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42453
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:44.738462 25262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:44.738642 25262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:44.738672 25262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:44.739822 25262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:44.739861 25262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:44.739881 25262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:44.739897 25262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:44.739916 25262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:44.741379 25262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:44.741609 25262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:44.741806 25262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:44.746117 25274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:44.746127 25270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:44.746227 25262 server_base.cc:1034] running on GCE node
W20260809 18:55:44.746169 25271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:44.746527 25262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:44.746774 25262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:44.747915 25262 hybrid_clock.cc:648] HybridClock initialized: now 1786301744747902 us; error 27 us; skew 500 ppm
Aug 09 18:55:44 dist-test-slave-6zbq krb5kdc[25240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301744, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:44.753142 25262 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:44.753549 25262 webserver.cc:460] Webserver started at http://127.0.0.1:36715/ using document root <none> and password file <none>
I20260809 18:55:44.753747 25262 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:44.753794 25262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:44.753913 25262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:44.754876 25262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "c8cd19239f7c4f86a984d088dbbb7b9e"
format_stamp: "Formatted at 2026-08-09 18:55:44 on dist-test-slave-6zbq"
server_key: "cc59078a200e1b876b0788f561a28713"
server_key_iv: "99e634283ba7bd76267be15697f2d0ec"
server_key_version: "encryptionkey@0"
I20260809 18:55:44.755178 25262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "c8cd19239f7c4f86a984d088dbbb7b9e"
format_stamp: "Formatted at 2026-08-09 18:55:44 on dist-test-slave-6zbq"
server_key: "cc59078a200e1b876b0788f561a28713"
server_key_iv: "99e634283ba7bd76267be15697f2d0ec"
server_key_version: "encryptionkey@0"
I20260809 18:55:44.756343 25262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:55:44.756947 25285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:44.757093 25262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:44.757172 25262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "c8cd19239f7c4f86a984d088dbbb7b9e"
format_stamp: "Formatted at 2026-08-09 18:55:44 on dist-test-slave-6zbq"
server_key: "cc59078a200e1b876b0788f561a28713"
server_key_iv: "99e634283ba7bd76267be15697f2d0ec"
server_key_version: "encryptionkey@0"
I20260809 18:55:44.757227 25262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:44.776399 25262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:44.777912 25262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:44.778039 25262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:44.781767 25262 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42453
I20260809 18:55:44.781803 25366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42453 every 8 connection(s)
I20260809 18:55:44.782094 25262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:44.782896 25369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:44.784999 25369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e: Bootstrap starting.
I20260809 18:55:44.785223 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25262
I20260809 18:55:44.785331 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:44.785501 24427 external_mini_cluster.cc:1428] Setting key e6732da00a2431ad412da2df4b88ad39
I20260809 18:55:44.785612 25369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:44.785916 25369 log.cc:824] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:44.786612 25369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e: No bootstrap required, opened a new log
I20260809 18:55:44.788617 25369 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8cd19239f7c4f86a984d088dbbb7b9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42453 } }
I20260809 18:55:44.788774 25369 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:44.788805 25369 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8cd19239f7c4f86a984d088dbbb7b9e, State: Initialized, Role: FOLLOWER
I20260809 18:55:44.788924 25369 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8cd19239f7c4f86a984d088dbbb7b9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42453 } }
I20260809 18:55:44.789006 25369 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:44.789036 25369 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:44.789084 25369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:44.789625 25369 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8cd19239f7c4f86a984d088dbbb7b9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42453 } }
I20260809 18:55:44.789738 25369 leader_election.cc:302] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8cd19239f7c4f86a984d088dbbb7b9e; no voters:
I20260809 18:55:44.789892 25369 leader_election.cc:288] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:44.789923 25379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:44.790108 25379 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [term 1 LEADER]: Becoming Leader. State: Replica: c8cd19239f7c4f86a984d088dbbb7b9e, State: Running, Role: LEADER
I20260809 18:55:44.790166 25369 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:44.790227 25379 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8cd19239f7c4f86a984d088dbbb7b9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42453 } }
I20260809 18:55:44.790557 25379 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8cd19239f7c4f86a984d088dbbb7b9e. Latest consensus state: current_term: 1 leader_uuid: "c8cd19239f7c4f86a984d088dbbb7b9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8cd19239f7c4f86a984d088dbbb7b9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42453 } } }
I20260809 18:55:44.790753 25379 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:44.790963 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:44.785934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44993 (local address 127.0.0.1:42453)
0809 18:55:44.786122 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:44.786131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:44.786165 (+ 34us) server_negotiation.cc:407] Connection header received
0809 18:55:44.786657 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:44.786667 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:44.786727 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:44.786837 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:44.787052 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:44.787897 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:44.788975 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:44.789445 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:44.789519 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:44.790215 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:44.790220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:44.790220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:44.790437 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:44.790474 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:44.790488 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:44.790527 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:44.790660 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 18:55:44.790731 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":60,"threads_started":1}
I20260809 18:55:44.790694 25380 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8cd19239f7c4f86a984d088dbbb7b9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8cd19239f7c4f86a984d088dbbb7b9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42453 } } }
I20260809 18:55:44.791092 25380 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:44.791231 25391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:44.791705 25391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:44.792004 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:44.793114 25391 catalog_manager.cc:1347] Generated new cluster ID: fbd9e62598754d489b98b122ce30f43c
I20260809 18:55:44.793154 25391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:44.796471 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:44.793426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51585 (local address 127.0.0.1:42453)
0809 18:55:44.793459 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:44.793460 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:44.793530 (+ 70us) server_negotiation.cc:407] Connection header received
0809 18:55:44.793590 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:44.793591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:44.793600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:44.793649 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:44.793749 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:44.794455 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:44.795364 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:44.795654 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:44.795726 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:44.796262 (+ 536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:44.796264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:44.796264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:44.796269 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:44.796283 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:44.796287 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:44.796339 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:44.796385 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 18:55:44.796405 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:55:44.801225 25391 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:44.801667 25391 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:44.806187 25391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8cd19239f7c4f86a984d088dbbb7b9e: Generated new TSK 0
I20260809 18:55:44.806375 25391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:44.812340 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25262
2026-08-09T18:55:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:44 dist-test-slave-6zbq krb5kdc[25418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:44 dist-test-slave-6zbq krb5kdc[25418](info): set up 2 sockets
Aug 09 18:55:44 dist-test-slave-6zbq krb5kdc[25418](info): commencing operation
krb5kdc: starting...
W20260809 18:55:45.955083 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:46 dist-test-slave-6zbq krb5kdc[25418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:46.058986 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:46.137405 25438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:46.137579 25438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:46.137600 25438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:46.138705 25438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:46.138742 25438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:46.138754 25438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:46.138765 25438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:46.138782 25438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:46.140151 25438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:46.140319 25438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:46.140473 25438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:46.144874 25443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:46.144879 25448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:46.144922 25438 server_base.cc:1034] running on GCE node
W20260809 18:55:46.144892 25445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:46.145270 25438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:46.145512 25438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:46.146636 25438 hybrid_clock.cc:648] HybridClock initialized: now 1786301746146627 us; error 26 us; skew 500 ppm
Aug 09 18:55:46 dist-test-slave-6zbq krb5kdc[25418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301746, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:46.148881 25438 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:46.149271 25438 webserver.cc:460] Webserver started at http://127.0.0.1:40437/ using document root <none> and password file <none>
I20260809 18:55:46.149451 25438 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:46.149493 25438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:46.149610 25438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:46.150599 25438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "7de361d3dd994b80a59df96862d143a0"
format_stamp: "Formatted at 2026-08-09 18:55:46 on dist-test-slave-6zbq"
server_key: "8510034d6d84f2a9d0b83586f7cdc72d"
server_key_iv: "b48b1d887e50efb64ce2150bd9a3408e"
server_key_version: "encryptionkey@0"
I20260809 18:55:46.150913 25438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "7de361d3dd994b80a59df96862d143a0"
format_stamp: "Formatted at 2026-08-09 18:55:46 on dist-test-slave-6zbq"
server_key: "8510034d6d84f2a9d0b83586f7cdc72d"
server_key_iv: "b48b1d887e50efb64ce2150bd9a3408e"
server_key_version: "encryptionkey@0"
I20260809 18:55:46.152064 25438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:55:46.152721 25460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:46.152876 25438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:46.152953 25438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "7de361d3dd994b80a59df96862d143a0"
format_stamp: "Formatted at 2026-08-09 18:55:46 on dist-test-slave-6zbq"
server_key: "8510034d6d84f2a9d0b83586f7cdc72d"
server_key_iv: "b48b1d887e50efb64ce2150bd9a3408e"
server_key_version: "encryptionkey@0"
I20260809 18:55:46.153005 25438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:46.156028 25438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:46.157711 25438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:46.157807 25438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:46.161387 25438 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33975
I20260809 18:55:46.161420 25546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33975 every 8 connection(s)
I20260809 18:55:46.161682 25438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:46.162580 25548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:46.162730 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25438
I20260809 18:55:46.162847 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:46.162999 24427 external_mini_cluster.cc:1428] Setting key af3a296747aed883fa921facdde7ed07
I20260809 18:55:46.165071 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:46.163412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43649 (local address 127.0.0.1:33975)
0809 18:55:46.163570 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:46.163579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:46.163611 (+ 32us) server_negotiation.cc:407] Connection header received
0809 18:55:46.164023 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:46.164033 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:46.164066 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:46.164209 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:46.164684 (+ 475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:46.164688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:46.164688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:46.164692 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:46.164735 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:46.164758 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:46.164829 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:46.164868 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 18:55:46.164905 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260809 18:55:46.165733 25548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0: Bootstrap starting.
I20260809 18:55:46.166034 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:46.166412 25548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:46.166666 25548 log.cc:824] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:46.167354 25548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0: No bootstrap required, opened a new log
I20260809 18:55:46.168190 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:46.167373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36991 (local address 127.0.0.1:33975)
0809 18:55:46.167414 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:46.167415 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:46.167473 (+ 58us) server_negotiation.cc:407] Connection header received
0809 18:55:46.167542 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:46.167542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:46.167546 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:46.167588 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:46.168031 (+ 443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:46.168032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:46.168032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:46.168036 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:46.168046 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:46.168050 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:46.168082 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:46.168134 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 18:55:46.168143 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 18:55:46.169351 25548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de361d3dd994b80a59df96862d143a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33975 } }
I20260809 18:55:46.169473 25548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:46.169505 25548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7de361d3dd994b80a59df96862d143a0, State: Initialized, Role: FOLLOWER
I20260809 18:55:46.169617 25548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de361d3dd994b80a59df96862d143a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33975 } }
I20260809 18:55:46.169687 25548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:46.169718 25548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:46.169765 25548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:46.170292 25548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de361d3dd994b80a59df96862d143a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33975 } }
I20260809 18:55:46.170394 25548 leader_election.cc:302] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7de361d3dd994b80a59df96862d143a0; no voters:
I20260809 18:55:46.170521 25548 leader_election.cc:288] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:46.170563 25567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:46.170683 25548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:46.170703 25567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [term 1 LEADER]: Becoming Leader. State: Replica: 7de361d3dd994b80a59df96862d143a0, State: Running, Role: LEADER
I20260809 18:55:46.170812 25567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de361d3dd994b80a59df96862d143a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33975 } }
I20260809 18:55:46.171119 25570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7de361d3dd994b80a59df96862d143a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de361d3dd994b80a59df96862d143a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33975 } } }
I20260809 18:55:46.171131 25571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7de361d3dd994b80a59df96862d143a0. Latest consensus state: current_term: 1 leader_uuid: "7de361d3dd994b80a59df96862d143a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7de361d3dd994b80a59df96862d143a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33975 } } }
I20260809 18:55:46.171334 25570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:46.171348 25571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:46.171725 25578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:46.172144 25578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:46.173492 25578 catalog_manager.cc:1347] Generated new cluster ID: e77822dcdba244d9b0d2dd30daea555a
I20260809 18:55:46.173540 25578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:46.179978 25578 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:46.180106 25578 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:46.182675 25578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7de361d3dd994b80a59df96862d143a0: Generated new TSK 0
I20260809 18:55:46.182843 25578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:46.189563 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25438
2026-08-09T18:55:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1370 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:46 dist-test-slave-6zbq krb5kdc[25599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:46 dist-test-slave-6zbq krb5kdc[25599](info): set up 2 sockets
Aug 09 18:55:46 dist-test-slave-6zbq krb5kdc[25599](info): commencing operation
krb5kdc: starting...
W20260809 18:55:47.327603 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:47 dist-test-slave-6zbq krb5kdc[25599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:47.434329 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:47.512624 25617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:47.512799 25617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:47.512818 25617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:47.513903 25617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:47.513943 25617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:47.513955 25617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:47.513978 25617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:47.513995 25617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:47.515463 25617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:47.515668 25617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:47.515857 25617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:47.520213 25629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:47.520274 25630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:47.520243 25632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:47.520292 25617 server_base.cc:1034] running on GCE node
I20260809 18:55:47.520625 25617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:47.520860 25617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:47.521991 25617 hybrid_clock.cc:648] HybridClock initialized: now 1786301747521978 us; error 29 us; skew 500 ppm
Aug 09 18:55:47 dist-test-slave-6zbq krb5kdc[25599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301747, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:47.524268 25617 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:47.524657 25617 webserver.cc:460] Webserver started at http://127.0.0.1:45993/ using document root <none> and password file <none>
I20260809 18:55:47.524868 25617 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:47.524909 25617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:47.525020 25617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:47.525951 25617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "6c3b495ab8e042588e5a0efae81bd372"
format_stamp: "Formatted at 2026-08-09 18:55:47 on dist-test-slave-6zbq"
server_key: "1a4443b7e9ca77822bb4d6f754d19adf"
server_key_iv: "c5e6535c0dd0218fbe307329c929c864"
server_key_version: "encryptionkey@0"
I20260809 18:55:47.526371 25617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "6c3b495ab8e042588e5a0efae81bd372"
format_stamp: "Formatted at 2026-08-09 18:55:47 on dist-test-slave-6zbq"
server_key: "1a4443b7e9ca77822bb4d6f754d19adf"
server_key_iv: "c5e6535c0dd0218fbe307329c929c864"
server_key_version: "encryptionkey@0"
I20260809 18:55:47.527628 25617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:55:47.528324 25641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:47.528469 25617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:55:47.528548 25617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "6c3b495ab8e042588e5a0efae81bd372"
format_stamp: "Formatted at 2026-08-09 18:55:47 on dist-test-slave-6zbq"
server_key: "1a4443b7e9ca77822bb4d6f754d19adf"
server_key_iv: "c5e6535c0dd0218fbe307329c929c864"
server_key_version: "encryptionkey@0"
I20260809 18:55:47.528613 25617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:47.535897 25617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:47.537388 25617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:47.537503 25617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:47.541165 25617 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33665
I20260809 18:55:47.541196 25714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33665 every 8 connection(s)
I20260809 18:55:47.541486 25617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:47.542236 25715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:47.544219 25715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372: Bootstrap starting.
I20260809 18:55:47.544781 25715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:47.545038 25715 log.cc:824] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:47.545607 25715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372: No bootstrap required, opened a new log
I20260809 18:55:47.547156 25715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3b495ab8e042588e5a0efae81bd372" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } }
I20260809 18:55:47.547283 25715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:47.547305 25715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3b495ab8e042588e5a0efae81bd372, State: Initialized, Role: FOLLOWER
I20260809 18:55:47.547400 25715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3b495ab8e042588e5a0efae81bd372" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } }
I20260809 18:55:47.547446 25715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:47.547470 25715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:47.547499 25715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:47.547941 25715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3b495ab8e042588e5a0efae81bd372" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } }
I20260809 18:55:47.548022 25715 leader_election.cc:302] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c3b495ab8e042588e5a0efae81bd372; no voters:
I20260809 18:55:47.548141 25715 leader_election.cc:288] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:47.548183 25723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:47.548295 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25617
I20260809 18:55:47.548331 25723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [term 1 LEADER]: Becoming Leader. State: Replica: 6c3b495ab8e042588e5a0efae81bd372, State: Running, Role: LEADER
I20260809 18:55:47.548378 25715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:47.548425 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:47.548416 25723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3b495ab8e042588e5a0efae81bd372" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } }
I20260809 18:55:47.548642 24427 external_mini_cluster.cc:1428] Setting key 306e699dc3e05da8019efcdd7efbb0f5
I20260809 18:55:47.548745 25723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c3b495ab8e042588e5a0efae81bd372. Latest consensus state: current_term: 1 leader_uuid: "6c3b495ab8e042588e5a0efae81bd372" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3b495ab8e042588e5a0efae81bd372" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } } }
I20260809 18:55:47.548823 25723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:47.548957 25728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c3b495ab8e042588e5a0efae81bd372" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3b495ab8e042588e5a0efae81bd372" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33665 } } }
I20260809 18:55:47.549072 25728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:47.549562 25738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:47.550251 25738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:47.551734 25738 catalog_manager.cc:1347] Generated new cluster ID: fae882fd686b4c429a8012dbf04f3d08
I20260809 18:55:47.551826 25738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:55:47 dist-test-slave-6zbq krb5kdc[25599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 18:55:47.558024 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:47.549187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:59371 (local address 127.0.0.1:33665)
0809 18:55:47.549417 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:47.549426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:47.549459 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:55:47.549813 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:47.549819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:47.549873 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:47.550003 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:47.550657 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:47.551837 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:47.552536 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:47.552956 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:47.553056 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:47.555166 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:47.555170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:47.555170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:47.555180 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:47.556624 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:47.556974 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:47.556976 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:47.556977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:47.557064 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:47.557269 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:47.557270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:47.557270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:47.557489 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:47.557648 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:47.557771 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 18:55:47.557869 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260809 18:55:47.558974 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:47.565614 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:47.560282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:35993 (local address 127.0.0.1:33665)
0809 18:55:47.560318 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:47.560320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:47.560437 (+ 117us) server_negotiation.cc:407] Connection header received
0809 18:55:47.560487 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:47.560488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:47.560497 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:47.560540 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:47.561003 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:47.561478 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:47.562126 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:47.562381 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:47.562437 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:47.563438 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:47.563439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:47.563440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:47.563446 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:47.564597 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:47.564916 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:47.564917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:47.564918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:47.564967 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:47.565174 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:47.565174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:47.565175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:47.565281 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:47.565374 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:47.565484 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 18:55:47.565539 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:55:47.570212 25738 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:47.570694 25738 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:47.577764 25738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c3b495ab8e042588e5a0efae81bd372: Generated new TSK 0
I20260809 18:55:47.577905 25738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:47.584285 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25617
W20260809 18:55:47.594367 25597 connection.cc:535] client connection to 127.0.0.1:33665 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33665): Connection reset by peer (error 104)
2026-08-09T18:55:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:47 dist-test-slave-6zbq krb5kdc[25774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:47 dist-test-slave-6zbq krb5kdc[25774](info): set up 2 sockets
Aug 09 18:55:47 dist-test-slave-6zbq krb5kdc[25774](info): commencing operation
krb5kdc: starting...
W20260809 18:55:48.717710 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:48 dist-test-slave-6zbq krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:48.822404 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:48.899650 25795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:48.899832 25795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:48.899863 25795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:48.900938 25795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:48.900978 25795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:48.900998 25795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:48.901013 25795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:48.901039 25795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:48.902532 25795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:48.902753 25795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:48.902940 25795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:48.907387 25803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:48.907393 25806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:48.907409 25804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:48.907410 25795 server_base.cc:1034] running on GCE node
I20260809 18:55:48.907699 25795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:48.907914 25795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:48.909044 25795 hybrid_clock.cc:648] HybridClock initialized: now 1786301748909031 us; error 29 us; skew 500 ppm
Aug 09 18:55:48 dist-test-slave-6zbq krb5kdc[25774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301748, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:48.911221 25795 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:48.911618 25795 webserver.cc:460] Webserver started at http://127.0.0.1:36631/ using document root <none> and password file <none>
I20260809 18:55:48.911801 25795 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:48.911847 25795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:48.911965 25795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:48.912891 25795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "2f169e65bc4c4bba80ec6188f483ff47"
format_stamp: "Formatted at 2026-08-09 18:55:48 on dist-test-slave-6zbq"
server_key: "3bbca663aa6cf3229b827f4b07ce1a03"
server_key_iv: "37a4f04cd0e2ef791e2c6161c94919cc"
server_key_version: "encryptionkey@0"
I20260809 18:55:48.913195 25795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "2f169e65bc4c4bba80ec6188f483ff47"
format_stamp: "Formatted at 2026-08-09 18:55:48 on dist-test-slave-6zbq"
server_key: "3bbca663aa6cf3229b827f4b07ce1a03"
server_key_iv: "37a4f04cd0e2ef791e2c6161c94919cc"
server_key_version: "encryptionkey@0"
I20260809 18:55:48.914381 25795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 18:55:48.915001 25815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:48.915150 25795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:55:48.915231 25795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "2f169e65bc4c4bba80ec6188f483ff47"
format_stamp: "Formatted at 2026-08-09 18:55:48 on dist-test-slave-6zbq"
server_key: "3bbca663aa6cf3229b827f4b07ce1a03"
server_key_iv: "37a4f04cd0e2ef791e2c6161c94919cc"
server_key_version: "encryptionkey@0"
I20260809 18:55:48.915302 25795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:48.946633 25795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:48.948292 25795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:48.948423 25795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:48.952205 25795 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43195
I20260809 18:55:48.952250 25908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43195 every 8 connection(s)
I20260809 18:55:48.952549 25795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:48.953325 25909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:48.955467 25909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47: Bootstrap starting.
I20260809 18:55:48.956014 25909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:48.956259 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25795
I20260809 18:55:48.956280 25909 log.cc:824] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:48.956364 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:48.956559 24427 external_mini_cluster.cc:1428] Setting key 11968c498046d908b1a855612de43029
I20260809 18:55:48.957332 25909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47: No bootstrap required, opened a new log
I20260809 18:55:48.959473 25909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f169e65bc4c4bba80ec6188f483ff47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260809 18:55:48.959637 25909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:48.959667 25909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f169e65bc4c4bba80ec6188f483ff47, State: Initialized, Role: FOLLOWER
I20260809 18:55:48.959781 25909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f169e65bc4c4bba80ec6188f483ff47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260809 18:55:48.959849 25909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:48.959883 25909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:48.959926 25909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:48.960429 25909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f169e65bc4c4bba80ec6188f483ff47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260809 18:55:48.960537 25909 leader_election.cc:302] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f169e65bc4c4bba80ec6188f483ff47; no voters:
I20260809 18:55:48.960706 25909 leader_election.cc:288] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:48.960741 25916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:48.960875 25916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [term 1 LEADER]: Becoming Leader. State: Replica: 2f169e65bc4c4bba80ec6188f483ff47, State: Running, Role: LEADER
I20260809 18:55:48.960948 25909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:48.961062 25916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f169e65bc4c4bba80ec6188f483ff47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260809 18:55:48.961481 25918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f169e65bc4c4bba80ec6188f483ff47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f169e65bc4c4bba80ec6188f483ff47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } } }
I20260809 18:55:48.961520 25923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f169e65bc4c4bba80ec6188f483ff47. Latest consensus state: current_term: 1 leader_uuid: "2f169e65bc4c4bba80ec6188f483ff47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f169e65bc4c4bba80ec6188f483ff47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } } }
I20260809 18:55:48.961568 25918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:48.961596 25923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:48.961783 25933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:48.962046 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:48.956992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:58893 (local address 127.0.0.1:43195)
0809 18:55:48.957144 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:48.957153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:48.957179 (+ 26us) server_negotiation.cc:407] Connection header received
0809 18:55:48.957619 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:48.957626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:48.957678 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:48.957794 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:48.958037 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:48.958952 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:48.960020 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:48.960406 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:48.960481 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:48.961212 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:48.961217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:48.961217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:48.961407 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:48.961448 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:48.961463 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:48.961782 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:48.961827 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 18:55:48.961852 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260809 18:55:48.964000 25933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:48.965416 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:48.965462 25933 catalog_manager.cc:1347] Generated new cluster ID: e7154d28190a4427a30ad639fd55efb0
I20260809 18:55:48.965505 25933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:48.973102 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:48.970587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:37911 (local address 127.0.0.1:43195)
0809 18:55:48.970620 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:48.970622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:48.970752 (+ 130us) server_negotiation.cc:407] Connection header received
0809 18:55:48.970792 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:48.970793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:48.970802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:48.970845 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:48.970973 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:48.971421 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:48.972078 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:48.972301 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:48.972354 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:48.972892 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:48.972893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:48.972893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:48.972898 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:48.972912 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:48.972916 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:48.972966 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:48.973029 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 18:55:48.973037 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:55:48.974129 25933 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:48.974644 25933 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:48.982467 25933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f169e65bc4c4bba80ec6188f483ff47: Generated new TSK 0
I20260809 18:55:48.982615 25933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:48.984807 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25795
W20260809 18:55:49.007408 25771 connection.cc:535] client connection to 127.0.0.1:43195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43195): Connection reset by peer (error 104)
2026-08-09T18:55:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1413 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:49 dist-test-slave-6zbq krb5kdc[25968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:49 dist-test-slave-6zbq krb5kdc[25968](info): set up 2 sockets
Aug 09 18:55:49 dist-test-slave-6zbq krb5kdc[25968](info): commencing operation
krb5kdc: starting...
W20260809 18:55:50.135676 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:50 dist-test-slave-6zbq krb5kdc[25968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:50.242694 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:50.320425 25985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:50.320606 25985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:50.320636 25985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:50.321766 25985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:50.321807 25985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:50.321826 25985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:50.321841 25985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:50.321857 25985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:50.323293 25985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:50.323513 25985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:50.323701 25985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:50.328115 25992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:50.328088 25995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:50.328172 25985 server_base.cc:1034] running on GCE node
W20260809 18:55:50.328097 25993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:50.328500 25985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:50.328760 25985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:50.329892 25985 hybrid_clock.cc:648] HybridClock initialized: now 1786301750329879 us; error 28 us; skew 500 ppm
Aug 09 18:55:50 dist-test-slave-6zbq krb5kdc[25968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301750, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:50.335254 25985 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:50.335644 25985 webserver.cc:460] Webserver started at http://127.0.0.1:36025/ using document root <none> and password file <none>
I20260809 18:55:50.335830 25985 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:50.335876 25985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:50.335994 25985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:50.336935 25985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "c1dacf5369974ce587ff87d287e82041"
format_stamp: "Formatted at 2026-08-09 18:55:50 on dist-test-slave-6zbq"
server_key: "805b5fc0a0d1562a27d43a2e33e1d3c2"
server_key_iv: "b4c71da3626955a486048bd1c736511a"
server_key_version: "encryptionkey@0"
I20260809 18:55:50.337244 25985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "c1dacf5369974ce587ff87d287e82041"
format_stamp: "Formatted at 2026-08-09 18:55:50 on dist-test-slave-6zbq"
server_key: "805b5fc0a0d1562a27d43a2e33e1d3c2"
server_key_iv: "b4c71da3626955a486048bd1c736511a"
server_key_version: "encryptionkey@0"
I20260809 18:55:50.338434 25985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:50.339098 26003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:50.339246 25985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:50.339327 25985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "c1dacf5369974ce587ff87d287e82041"
format_stamp: "Formatted at 2026-08-09 18:55:50 on dist-test-slave-6zbq"
server_key: "805b5fc0a0d1562a27d43a2e33e1d3c2"
server_key_iv: "b4c71da3626955a486048bd1c736511a"
server_key_version: "encryptionkey@0"
I20260809 18:55:50.339392 25985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:50.342423 25985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:50.344316 25985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:50.344439 25985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:50.348106 25985 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39245
I20260809 18:55:50.348137 26079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39245 every 8 connection(s)
I20260809 18:55:50.348400 25985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:50.349290 26080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:50.351301 26080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041: Bootstrap starting.
I20260809 18:55:50.351809 26080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:50.352049 26080 log.cc:824] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:50.352643 26080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041: No bootstrap required, opened a new log
I20260809 18:55:50.354113 26080 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1dacf5369974ce587ff87d287e82041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260809 18:55:50.354245 26080 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:50.354279 26080 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1dacf5369974ce587ff87d287e82041, State: Initialized, Role: FOLLOWER
I20260809 18:55:50.354439 26080 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1dacf5369974ce587ff87d287e82041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260809 18:55:50.354518 26080 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:50.354550 26080 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:50.354596 26080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:50.355058 26080 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1dacf5369974ce587ff87d287e82041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260809 18:55:50.355144 26080 leader_election.cc:302] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1dacf5369974ce587ff87d287e82041; no voters:
I20260809 18:55:50.355296 26080 leader_election.cc:288] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:50.355334 26086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:50.355500 26086 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [term 1 LEADER]: Becoming Leader. State: Replica: c1dacf5369974ce587ff87d287e82041, State: Running, Role: LEADER
I20260809 18:55:50.355530 26080 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:50.355628 26086 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1dacf5369974ce587ff87d287e82041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } }
I20260809 18:55:50.355949 26087 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1dacf5369974ce587ff87d287e82041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1dacf5369974ce587ff87d287e82041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } } }
I20260809 18:55:50.356046 26087 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:50.355952 26093 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1dacf5369974ce587ff87d287e82041. Latest consensus state: current_term: 1 leader_uuid: "c1dacf5369974ce587ff87d287e82041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1dacf5369974ce587ff87d287e82041" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39245 } } }
I20260809 18:55:50.356168 26093 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:50.356434 26098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:50.356477 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 25985
I20260809 18:55:50.356614 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:50.356814 24427 external_mini_cluster.cc:1428] Setting key aa7175ea8afb7c000dfe100419cbf9e8
I20260809 18:55:50.357241 26098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:50.358578 26098 catalog_manager.cc:1347] Generated new cluster ID: 5c060235feb24d6da06ca22c98b68da5
I20260809 18:55:50.358623 26098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:50.359786 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:50.357214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:58339 (local address 127.0.0.1:39245)
0809 18:55:50.358475 (+ 1261us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:50.358483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:50.358512 (+ 29us) server_negotiation.cc:407] Connection header received
0809 18:55:50.358615 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:50.358622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:50.358652 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:50.358805 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:50.359401 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:50.359404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:50.359405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:50.359409 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:50.359446 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:50.359459 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:50.359518 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:50.359590 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 18:55:50.359626 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1211,"thread_start_us":69,"threads_started":1}
I20260809 18:55:50.360561 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:50.362905 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:50.361984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:46689 (local address 127.0.0.1:39245)
0809 18:55:50.362018 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:50.362020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:50.362142 (+ 122us) server_negotiation.cc:407] Connection header received
0809 18:55:50.362182 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:50.362183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:50.362188 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:50.362251 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:50.362710 (+ 459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:50.362711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:50.362712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:50.362715 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:50.362729 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:50.362733 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:50.362793 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:50.362838 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 18:55:50.362849 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:55:50.377331 26098 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:50.377470 26098 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:50.381826 26098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1dacf5369974ce587ff87d287e82041: Generated new TSK 0
I20260809 18:55:50.381965 26098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:50.386456 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 25985
2026-08-09T18:55:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:50 dist-test-slave-6zbq krb5kdc[26137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:50 dist-test-slave-6zbq krb5kdc[26137](info): set up 2 sockets
Aug 09 18:55:50 dist-test-slave-6zbq krb5kdc[26137](info): commencing operation
krb5kdc: starting...
W20260809 18:55:51.532728 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:51 dist-test-slave-6zbq krb5kdc[26137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:51.640244 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44455
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:51.717856 26157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:51.718045 26157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:51.718066 26157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:51.719148 26157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:51.719187 26157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:51.719200 26157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:51.719211 26157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:51.719221 26157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:51.720616 26157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44455
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:51.720791 26157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:51.720956 26157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:51.725335 26176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:51.725400 26179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:51.725400 26177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:51.725425 26157 server_base.cc:1034] running on GCE node
I20260809 18:55:51.725761 26157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:51.726020 26157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:51.727144 26157 hybrid_clock.cc:648] HybridClock initialized: now 1786301751727132 us; error 33 us; skew 500 ppm
Aug 09 18:55:51 dist-test-slave-6zbq krb5kdc[26137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301751, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:51.729347 26157 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:51.729748 26157 webserver.cc:460] Webserver started at http://127.0.0.1:44541/ using document root <none> and password file <none>
I20260809 18:55:51.729951 26157 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:51.729996 26157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:51.730114 26157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:51.731109 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8"
format_stamp: "Formatted at 2026-08-09 18:55:51 on dist-test-slave-6zbq"
server_key: "ff3917d334a3864aa407eb716c3e1b85"
server_key_iv: "138b40352717862e58d6e74ee3752519"
server_key_version: "encryptionkey@0"
I20260809 18:55:51.731428 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8"
format_stamp: "Formatted at 2026-08-09 18:55:51 on dist-test-slave-6zbq"
server_key: "ff3917d334a3864aa407eb716c3e1b85"
server_key_iv: "138b40352717862e58d6e74ee3752519"
server_key_version: "encryptionkey@0"
I20260809 18:55:51.732577 26157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:55:51.733175 26192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:51.733326 26157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:55:51.733407 26157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8"
format_stamp: "Formatted at 2026-08-09 18:55:51 on dist-test-slave-6zbq"
server_key: "ff3917d334a3864aa407eb716c3e1b85"
server_key_iv: "138b40352717862e58d6e74ee3752519"
server_key_version: "encryptionkey@0"
I20260809 18:55:51.733471 26157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:51.744925 26157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:51.746464 26157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:51.746587 26157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:51.750245 26157 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44455
I20260809 18:55:51.750283 26281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44455 every 8 connection(s)
I20260809 18:55:51.750612 26157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:51.751408 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:51.753506 26286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8: Bootstrap starting.
I20260809 18:55:51.753898 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 26157
I20260809 18:55:51.754019 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:51.754195 26286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:51.754220 24427 external_mini_cluster.cc:1428] Setting key d5133df91e89ac608e2dc15b461431af
I20260809 18:55:51.754539 26286 log.cc:824] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:51.755249 26286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8: No bootstrap required, opened a new log
I20260809 18:55:51.757015 26286 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44455 } }
I20260809 18:55:51.757143 26286 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:51.757175 26286 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9d1bf7ff21d445e8d7d8d3973c9d1a8, State: Initialized, Role: FOLLOWER
I20260809 18:55:51.757297 26286 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44455 } }
I20260809 18:55:51.757368 26286 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:51.757400 26286 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:51.757444 26286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:51.757954 26286 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44455 } }
I20260809 18:55:51.758049 26286 leader_election.cc:302] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9d1bf7ff21d445e8d7d8d3973c9d1a8; no voters:
I20260809 18:55:51.758193 26286 leader_election.cc:288] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:51.758235 26292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:51.758452 26286 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:51.758472 26292 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [term 1 LEADER]: Becoming Leader. State: Replica: c9d1bf7ff21d445e8d7d8d3973c9d1a8, State: Running, Role: LEADER
I20260809 18:55:51.758605 26292 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44455 } }
Aug 09 18:55:51 dist-test-slave-6zbq krb5kdc[26137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 18:55:51.759974 26297 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9d1bf7ff21d445e8d7d8d3973c9d1a8. Latest consensus state: current_term: 1 leader_uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44455 } } }
I20260809 18:55:51.760082 26295 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d1bf7ff21d445e8d7d8d3973c9d1a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44455 } } }
W20260809 18:55:51.760380 26317 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 18:55:51.760898 26295 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:51.760888 26297 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8 [sys.catalog]: This master's current role is: LEADER
W20260809 18:55:51.760913 26317 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:55:51.764114 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:51.754721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:35781 (local address 127.0.0.1:44455)
0809 18:55:51.754875 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:51.754884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:51.754913 (+ 29us) server_negotiation.cc:407] Connection header received
0809 18:55:51.755349 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:51.755357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:51.755412 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:51.755528 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:51.756114 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:51.756969 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:51.757799 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:51.758279 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:51.758714 (+ 435us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:51.761202 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:51.761206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:51.761206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:51.761217 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:51.762688 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:51.763107 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:51.763110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:51.763110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:51.763186 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:51.763401 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:51.763402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:51.763402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:51.763595 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:51.763719 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:51.763847 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 18:55:51.763918 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260809 18:55:51.765005 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:51.770444 26328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:51.770634 26328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:51.772171 26328 catalog_manager.cc:1347] Generated new cluster ID: 4f14f6b8b35940a1a89b53714973ca3a
I20260809 18:55:51.772276 26328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:51.772554 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:51.766263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:55675 (local address 127.0.0.1:44455)
0809 18:55:51.766297 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:51.766299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:51.766357 (+ 58us) server_negotiation.cc:407] Connection header received
0809 18:55:51.766440 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:51.766441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:51.766448 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:51.766490 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:51.767091 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:51.767528 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:51.768482 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:51.768682 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:51.768725 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:51.769949 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:51.769950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:51.769950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:51.769956 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:51.771211 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:51.771723 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:51.771724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:51.771725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:51.771781 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:51.772022 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:51.772023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:51.772023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:51.772148 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:51.772252 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:51.772418 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 18:55:51.772477 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:55:51.798055 26328 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:51.798565 26328 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:51.805284 26328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9d1bf7ff21d445e8d7d8d3973c9d1a8: Generated new TSK 0
I20260809 18:55:51.805433 26328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:51.809309 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 26157
W20260809 18:55:51.821975 26134 connection.cc:535] client connection to 127.0.0.1:44455 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44455): Connection reset by peer (error 104)
2026-08-09T18:55:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:51 dist-test-slave-6zbq krb5kdc[26340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:51 dist-test-slave-6zbq krb5kdc[26340](info): set up 2 sockets
Aug 09 18:55:51 dist-test-slave-6zbq krb5kdc[26340](info): commencing operation
krb5kdc: starting...
W20260809 18:55:52.964918 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:53 dist-test-slave-6zbq krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:53.069908 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:53.146517 26361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:53.146700 26361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:53.146730 26361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:53.147812 26361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:53.147853 26361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:53.147871 26361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:53.147887 26361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:53.147903 26361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:53.149344 26361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45237
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:53.149562 26361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:53.149756 26361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:53.154749 26376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:53.154765 26371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:53.154759 26372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:53.155051 26361 server_base.cc:1034] running on GCE node
I20260809 18:55:53.155270 26361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:53.155488 26361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:53.156628 26361 hybrid_clock.cc:648] HybridClock initialized: now 1786301753156617 us; error 28 us; skew 500 ppm
Aug 09 18:55:53 dist-test-slave-6zbq krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301753, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:53.158859 26361 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:53.159229 26361 webserver.cc:460] Webserver started at http://127.0.0.1:40599/ using document root <none> and password file <none>
I20260809 18:55:53.159413 26361 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:53.159459 26361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:53.159576 26361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:53.160513 26361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "bc9fe6d2284c47e9b953e8a02c914def"
format_stamp: "Formatted at 2026-08-09 18:55:53 on dist-test-slave-6zbq"
server_key: "5bbefe9185903048f8d3694306749e21"
server_key_iv: "7f55f17935ee7a5bf2ad0e70b629c52a"
server_key_version: "encryptionkey@0"
I20260809 18:55:53.160833 26361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "bc9fe6d2284c47e9b953e8a02c914def"
format_stamp: "Formatted at 2026-08-09 18:55:53 on dist-test-slave-6zbq"
server_key: "5bbefe9185903048f8d3694306749e21"
server_key_iv: "7f55f17935ee7a5bf2ad0e70b629c52a"
server_key_version: "encryptionkey@0"
I20260809 18:55:53.161980 26361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:55:53.162631 26388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:53.162781 26361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:53.162863 26361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "bc9fe6d2284c47e9b953e8a02c914def"
format_stamp: "Formatted at 2026-08-09 18:55:53 on dist-test-slave-6zbq"
server_key: "5bbefe9185903048f8d3694306749e21"
server_key_iv: "7f55f17935ee7a5bf2ad0e70b629c52a"
server_key_version: "encryptionkey@0"
I20260809 18:55:53.162922 26361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:53.169165 26361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:53.170722 26361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:53.170848 26361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:53.174501 26361 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45237
I20260809 18:55:53.174528 26475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45237 every 8 connection(s)
I20260809 18:55:53.174816 26361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:53.175545 26476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:53.177556 26476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def: Bootstrap starting.
I20260809 18:55:53.178074 26476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:53.178320 26476 log.cc:824] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:53.178958 26476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def: No bootstrap required, opened a new log
I20260809 18:55:53.180447 26476 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9fe6d2284c47e9b953e8a02c914def" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45237 } }
I20260809 18:55:53.180572 26476 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:53.180604 26476 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9fe6d2284c47e9b953e8a02c914def, State: Initialized, Role: FOLLOWER
I20260809 18:55:53.180745 26476 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9fe6d2284c47e9b953e8a02c914def" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45237 } }
I20260809 18:55:53.180820 26476 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:53.180852 26476 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:53.180897 26476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:53.181375 26476 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9fe6d2284c47e9b953e8a02c914def" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45237 } }
I20260809 18:55:53.181463 26476 leader_election.cc:302] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc9fe6d2284c47e9b953e8a02c914def; no voters:
I20260809 18:55:53.181614 26476 leader_election.cc:288] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:53.181643 26479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:53.181762 26479 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [term 1 LEADER]: Becoming Leader. State: Replica: bc9fe6d2284c47e9b953e8a02c914def, State: Running, Role: LEADER
I20260809 18:55:53.181844 26476 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:53.181905 26479 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9fe6d2284c47e9b953e8a02c914def" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45237 } }
I20260809 18:55:53.182247 26481 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc9fe6d2284c47e9b953e8a02c914def. Latest consensus state: current_term: 1 leader_uuid: "bc9fe6d2284c47e9b953e8a02c914def" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9fe6d2284c47e9b953e8a02c914def" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45237 } } }
I20260809 18:55:53.182358 26481 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:53.182371 26480 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc9fe6d2284c47e9b953e8a02c914def" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc9fe6d2284c47e9b953e8a02c914def" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45237 } } }
I20260809 18:55:53.182420 26480 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:53.182587 26492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:53.183120 26492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:53.183526 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 26361
I20260809 18:55:53.183655 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:53.183840 24427 external_mini_cluster.cc:1428] Setting key 7194d4bbafba1a62d2f943692c5eb40b
I20260809 18:55:53.184684 26492 catalog_manager.cc:1347] Generated new cluster ID: 76acf8b37f7043089b1f0789456efd8a
I20260809 18:55:53.184728 26492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:53.189565 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:53.184208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:41009 (local address 127.0.0.1:45237)
0809 18:55:53.184574 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:53.184583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:53.184612 (+ 29us) server_negotiation.cc:407] Connection header received
0809 18:55:53.184716 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:53.184723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:53.184791 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:53.184921 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:53.185161 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:53.186586 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:53.187650 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:53.188219 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:53.188307 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:53.188965 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:53.188969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:53.188969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:53.189145 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:53.189179 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:53.189192 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:53.189263 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:53.189361 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 18:55:53.189384 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":90,"threads_started":1}
I20260809 18:55:53.190555 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:53.196678 26492 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:53.197120 26492 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:53.205538 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:53.191953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:35879 (local address 127.0.0.1:45237)
0809 18:55:53.192005 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:53.192006 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:53.196421 (+ 4415us) server_negotiation.cc:407] Connection header received
0809 18:55:53.196593 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:53.196594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:53.196604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:53.196630 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:53.202569 (+ 5939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:53.203313 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:53.204256 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:53.204595 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:53.204657 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:53.205285 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:53.205286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:53.205287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:53.205290 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:53.205301 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:53.205304 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:53.205347 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:53.205459 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 18:55:53.205471 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 18:55:53.205607 26492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc9fe6d2284c47e9b953e8a02c914def: Generated new TSK 0
I20260809 18:55:53.205775 26492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:53.209806 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 26361
W20260809 18:55:53.223894 26338 connection.cc:535] client connection to 127.0.0.1:45237 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45237): Connection reset by peer (error 104)
2026-08-09T18:55:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:53 dist-test-slave-6zbq krb5kdc[26519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:53 dist-test-slave-6zbq krb5kdc[26519](info): set up 2 sockets
Aug 09 18:55:53 dist-test-slave-6zbq krb5kdc[26519](info): commencing operation
krb5kdc: starting...
W20260809 18:55:54.340699 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:54 dist-test-slave-6zbq krb5kdc[26519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 18:55:54.446179 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41955
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:54.524168 26539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:54.524359 26539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:54.524382 26539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:54.525437 26539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:54.525470 26539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:54.525482 26539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:54.525493 26539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:54.525503 26539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:54.526927 26539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41955
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:54.527112 26539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:54.527275 26539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:54.531677 26552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:54.531648 26547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:54.531649 26548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:54.531694 26539 server_base.cc:1034] running on GCE node
I20260809 18:55:54.532104 26539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:54.532348 26539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:54.533476 26539 hybrid_clock.cc:648] HybridClock initialized: now 1786301754533464 us; error 31 us; skew 500 ppm
Aug 09 18:55:54 dist-test-slave-6zbq krb5kdc[26519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:54.535693 26539 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 18:55:54.536108 26539 webserver.cc:460] Webserver started at http://127.0.0.1:46069/ using document root <none> and password file <none>
I20260809 18:55:54.536309 26539 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:54.536355 26539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:54.536473 26539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:54.537397 26539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "d53a9eff8cef4ead98f1c49887d78af5"
format_stamp: "Formatted at 2026-08-09 18:55:54 on dist-test-slave-6zbq"
server_key: "b4dea30cda8d4e8a41a9d6efa232eb90"
server_key_iv: "443a89533d1609fda390e489a7629761"
server_key_version: "encryptionkey@0"
I20260809 18:55:54.537704 26539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "d53a9eff8cef4ead98f1c49887d78af5"
format_stamp: "Formatted at 2026-08-09 18:55:54 on dist-test-slave-6zbq"
server_key: "b4dea30cda8d4e8a41a9d6efa232eb90"
server_key_iv: "443a89533d1609fda390e489a7629761"
server_key_version: "encryptionkey@0"
I20260809 18:55:54.538923 26539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:54.539620 26564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:54.539775 26539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:54.539858 26539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "d53a9eff8cef4ead98f1c49887d78af5"
format_stamp: "Formatted at 2026-08-09 18:55:54 on dist-test-slave-6zbq"
server_key: "b4dea30cda8d4e8a41a9d6efa232eb90"
server_key_iv: "443a89533d1609fda390e489a7629761"
server_key_version: "encryptionkey@0"
I20260809 18:55:54.539917 26539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:54.543038 26539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:54.544595 26539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:54.544714 26539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:54.548475 26539 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41955
I20260809 18:55:54.548503 26648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41955 every 8 connection(s)
I20260809 18:55:54.548784 26539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:54.549647 26649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:54.549876 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 26539
I20260809 18:55:54.550005 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:54.550200 24427 external_mini_cluster.cc:1428] Setting key 9ef48926f0a764a06b83fcc58818c1ba
I20260809 18:55:54.552296 26649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5: Bootstrap starting.
I20260809 18:55:54.552366 26653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:54.550674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:56033 (local address 127.0.0.1:41955)
0809 18:55:54.550849 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:54.550857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:54.550880 (+ 23us) server_negotiation.cc:407] Connection header received
0809 18:55:54.551255 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:54.551262 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:54.551285 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:54.551394 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:54.551971 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:54.551975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:54.551975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:54.551978 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:54.552012 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:54.552021 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:54.552074 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:54.552155 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 18:55:54.552186 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260809 18:55:54.552943 26649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:54.553251 26649 log.cc:824] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:54.553355 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:54.553952 26649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5: No bootstrap required, opened a new log
I20260809 18:55:54.555639 26653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:54.554816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.181:49211 (local address 127.0.0.1:41955)
0809 18:55:54.554852 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:54.554854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:54.554917 (+ 63us) server_negotiation.cc:407] Connection header received
0809 18:55:54.554983 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:54.554983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:54.554987 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:54.555028 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:54.555476 (+ 448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:54.555476 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:54.555477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:54.555480 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:54.555491 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:54.555494 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:54.555532 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:54.555581 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 18:55:54.555590 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:54.556360 26649 raft_consensus.cc:348] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53a9eff8cef4ead98f1c49887d78af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41955 } }
I20260809 18:55:54.556483 26649 raft_consensus.cc:374] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:54.556504 26649 raft_consensus.cc:729] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53a9eff8cef4ead98f1c49887d78af5, State: Initialized, Role: FOLLOWER
I20260809 18:55:54.556612 26649 consensus_queue.cc:260] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53a9eff8cef4ead98f1c49887d78af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41955 } }
I20260809 18:55:54.556689 26649 raft_consensus.cc:388] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:54.556720 26649 raft_consensus.cc:482] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:54.556764 26649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:54.557240 26649 raft_consensus.cc:504] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53a9eff8cef4ead98f1c49887d78af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41955 } }
I20260809 18:55:54.557329 26649 leader_election.cc:302] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d53a9eff8cef4ead98f1c49887d78af5; no voters:
I20260809 18:55:54.557473 26649 leader_election.cc:288] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:54.557515 26665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:54.557672 26665 raft_consensus.cc:686] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [term 1 LEADER]: Becoming Leader. State: Replica: d53a9eff8cef4ead98f1c49887d78af5, State: Running, Role: LEADER
I20260809 18:55:54.557680 26649 sys_catalog.cc:564] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:54.557814 26665 consensus_queue.cc:237] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53a9eff8cef4ead98f1c49887d78af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41955 } }
I20260809 18:55:54.558156 26666 sys_catalog.cc:455] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d53a9eff8cef4ead98f1c49887d78af5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53a9eff8cef4ead98f1c49887d78af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41955 } } }
I20260809 18:55:54.558296 26666 sys_catalog.cc:458] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:54.558212 26668 sys_catalog.cc:455] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d53a9eff8cef4ead98f1c49887d78af5. Latest consensus state: current_term: 1 leader_uuid: "d53a9eff8cef4ead98f1c49887d78af5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d53a9eff8cef4ead98f1c49887d78af5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41955 } } }
I20260809 18:55:54.558416 26668 sys_catalog.cc:458] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:54.558918 26677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:54.559361 26677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:54.560781 26677 catalog_manager.cc:1347] Generated new cluster ID: 6cbfb8be8fc149ac80f043946e1bb45d
I20260809 18:55:54.560830 26677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:54.568293 26677 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:54.568439 26677 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:54.577520 26677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d53a9eff8cef4ead98f1c49887d78af5: Generated new TSK 0
I20260809 18:55:54.577682 26677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:54.584534 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 26539
2026-08-09T18:55:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:54 dist-test-slave-6zbq krb5kdc[26694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:54 dist-test-slave-6zbq krb5kdc[26694](info): set up 2 sockets
Aug 09 18:55:54 dist-test-slave-6zbq krb5kdc[26694](info): commencing operation
krb5kdc: starting...
W20260809 18:55:55.732879 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:55 dist-test-slave-6zbq krb5kdc[26694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:55:55.838047 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:41879
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:44815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:41879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:55.915110 26716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:55.915293 26716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:55.915316 26716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:55.916366 26716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:55.916397 26716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:55.916409 26716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:55.916420 26716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:55.916435 26716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:55.917804 26716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:44815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:41879
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:41879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:55.917975 26716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:55.918128 26716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:55.922504 26725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:55.922509 26727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:55.922501 26729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:55.922607 26716 server_base.cc:1034] running on GCE node
I20260809 18:55:55.922880 26716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:55.923131 26716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:55.924271 26716 hybrid_clock.cc:648] HybridClock initialized: now 1786301755924256 us; error 28 us; skew 500 ppm
Aug 09 18:55:55 dist-test-slave-6zbq krb5kdc[26694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301755, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:55.930940 26716 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:55:55.931321 26716 webserver.cc:460] Webserver started at http://127.23.218.254:43465/ using document root <none> and password file <none>
I20260809 18:55:55.931519 26716 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:55.931565 26716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:55.931686 26716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:55.932622 26716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "6a8b242f4a4e4431a38d3e6ab2351990"
format_stamp: "Formatted at 2026-08-09 18:55:55 on dist-test-slave-6zbq"
server_key: "744e2e7270b22115c8f565f3c3513640"
server_key_iv: "32998643a7e6a35e07edd5f755237c89"
server_key_version: "encryptionkey@0"
I20260809 18:55:55.932934 26716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "6a8b242f4a4e4431a38d3e6ab2351990"
format_stamp: "Formatted at 2026-08-09 18:55:55 on dist-test-slave-6zbq"
server_key: "744e2e7270b22115c8f565f3c3513640"
server_key_iv: "32998643a7e6a35e07edd5f755237c89"
server_key_version: "encryptionkey@0"
I20260809 18:55:55.934087 26716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:55:55.934772 26736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:55.934918 26716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:55.934999 26716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "6a8b242f4a4e4431a38d3e6ab2351990"
format_stamp: "Formatted at 2026-08-09 18:55:55 on dist-test-slave-6zbq"
server_key: "744e2e7270b22115c8f565f3c3513640"
server_key_iv: "32998643a7e6a35e07edd5f755237c89"
server_key_version: "encryptionkey@0"
I20260809 18:55:55.935062 26716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:55.944027 26716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:55.945574 26716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:55.945703 26716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:55.949359 26716 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:41879
I20260809 18:55:55.949393 26825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:41879 every 8 connection(s)
I20260809 18:55:55.949679 26716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:55.950527 26828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:55.951613 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 26716
I20260809 18:55:55.951732 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:55.951911 24427 external_mini_cluster.cc:1428] Setting key 5e6404585a980b3fe2df4fd9e97b1c6a
I20260809 18:55:55.953028 26828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990: Bootstrap starting.
I20260809 18:55:55.953655 26828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:55.953969 26828 log.cc:824] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:55.954698 26828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990: No bootstrap required, opened a new log
I20260809 18:55:55.956831 26828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a8b242f4a4e4431a38d3e6ab2351990" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41879 } }
I20260809 18:55:55.956997 26828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:55.957046 26828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8b242f4a4e4431a38d3e6ab2351990, State: Initialized, Role: FOLLOWER
I20260809 18:55:55.957168 26828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a8b242f4a4e4431a38d3e6ab2351990" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41879 } }
I20260809 18:55:55.957242 26828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:55.957288 26828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:55.957334 26828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:55.957826 26828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a8b242f4a4e4431a38d3e6ab2351990" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41879 } }
I20260809 18:55:55.957934 26828 leader_election.cc:302] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a8b242f4a4e4431a38d3e6ab2351990; no voters:
I20260809 18:55:55.958091 26828 leader_election.cc:288] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:55.958153 26837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:55.958300 26837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [term 1 LEADER]: Becoming Leader. State: Replica: 6a8b242f4a4e4431a38d3e6ab2351990, State: Running, Role: LEADER
I20260809 18:55:55.958442 26828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 18:55:55 dist-test-slave-6zbq krb5kdc[26694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:55:55.958446 26837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a8b242f4a4e4431a38d3e6ab2351990" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41879 } }
I20260809 18:55:55.958891 26837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a8b242f4a4e4431a38d3e6ab2351990. Latest consensus state: current_term: 1 leader_uuid: "6a8b242f4a4e4431a38d3e6ab2351990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a8b242f4a4e4431a38d3e6ab2351990" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41879 } } }
I20260809 18:55:55.959012 26837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:55.959012 26840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a8b242f4a4e4431a38d3e6ab2351990" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a8b242f4a4e4431a38d3e6ab2351990" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 41879 } } }
I20260809 18:55:55.959159 26840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:55.959414 26853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:55.959923 26853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:55.961439 26853 catalog_manager.cc:1347] Generated new cluster ID: 4fdd579b0ca94314b04e7b484f2ea233
I20260809 18:55:55.961478 26853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:55.965068 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:55.952345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44027 (local address 127.23.218.254:41879)
0809 18:55:55.952529 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:55.952539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:55.952572 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:55:55.953072 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:55.953081 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:55.953141 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:55.953283 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:55.953813 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:55.955123 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:55.955852 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:55.956463 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:55.956582 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:55.959058 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:55.959063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:55.959063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:55.959074 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:55.963499 (+ 4425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:55.963925 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:55.963928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:55.963928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:55.964033 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:55.964266 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:55.964267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:55.964267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:55.964524 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:55.964660 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:55.964803 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 18:55:55.964892 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260809 18:55:55.966200 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:55.974076 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:55.967588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34643 (local address 127.23.218.254:41879)
0809 18:55:55.967622 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:55.967623 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:55.967770 (+ 147us) server_negotiation.cc:407] Connection header received
0809 18:55:55.967813 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:55.967814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:55.967823 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:55.967879 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:55.968412 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:55.969135 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:55.970047 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:55.970392 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:55.970458 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:55.971614 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:55.971616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:55.971616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:55:55.971622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:55.973016 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:55.973382 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:55.973383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:55.973383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:55.973439 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:55:55.973623 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:55:55.973624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:55:55.973624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:55:55.973741 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:55.973837 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:55.973934 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 18:55:55.973994 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:55:55.991400 26853 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:55.991859 26853 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:55.998035 26853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a8b242f4a4e4431a38d3e6ab2351990: Generated new TSK 0
I20260809 18:55:55.998189 26853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:56.006841 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 26716
W20260809 18:55:56.020320 26692 connection.cc:535] client connection to 127.23.218.254:41879 recv error: Network error: failed to read from TLS socket (remote: 127.23.218.254:41879): Connection reset by peer (error 104)
2026-08-09T18:55:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:56 dist-test-slave-6zbq krb5kdc[26891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:56 dist-test-slave-6zbq krb5kdc[26891](info): set up 2 sockets
Aug 09 18:55:56 dist-test-slave-6zbq krb5kdc[26891](info): commencing operation
krb5kdc: starting...
W20260809 18:55:57.140301 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:57 dist-test-slave-6zbq krb5kdc[26891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:55:57.251257 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:46201
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:46201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:57.333958 26912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:57.334147 26912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:57.334169 26912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:57.335306 26912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:57.335345 26912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:57.335358 26912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:57.335371 26912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:57.335386 26912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:57.336795 26912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:46201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:46201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:57.336971 26912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:57.337143 26912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:57.341730 26919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:57.341722 26922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:57.341791 26912 server_base.cc:1034] running on GCE node
W20260809 18:55:57.341722 26918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:57.342134 26912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:57.342442 26912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:57.343571 26912 hybrid_clock.cc:648] HybridClock initialized: now 1786301757343562 us; error 29 us; skew 500 ppm
Aug 09 18:55:57 dist-test-slave-6zbq krb5kdc[26891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301757, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:57.345790 26912 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:55:57.346197 26912 webserver.cc:460] Webserver started at http://127.23.218.254:43075/ using document root <none> and password file <none>
I20260809 18:55:57.346472 26912 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:57.346522 26912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:57.346648 26912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:57.347623 26912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "afec3d940183447fb3fe10d769be3a47"
format_stamp: "Formatted at 2026-08-09 18:55:57 on dist-test-slave-6zbq"
server_key: "baa5c9bd834d491c0aefe997c94a53e2"
server_key_iv: "9340b3498745530906a4ad18b772c792"
server_key_version: "encryptionkey@0"
I20260809 18:55:57.347959 26912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "afec3d940183447fb3fe10d769be3a47"
format_stamp: "Formatted at 2026-08-09 18:55:57 on dist-test-slave-6zbq"
server_key: "baa5c9bd834d491c0aefe997c94a53e2"
server_key_iv: "9340b3498745530906a4ad18b772c792"
server_key_version: "encryptionkey@0"
I20260809 18:55:57.349139 26912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:55:57.349752 26934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:57.349902 26912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:57.349982 26912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "afec3d940183447fb3fe10d769be3a47"
format_stamp: "Formatted at 2026-08-09 18:55:57 on dist-test-slave-6zbq"
server_key: "baa5c9bd834d491c0aefe997c94a53e2"
server_key_iv: "9340b3498745530906a4ad18b772c792"
server_key_version: "encryptionkey@0"
I20260809 18:55:57.350049 26912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:57.360496 26912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:57.362136 26912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:57.362257 26912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:57.365974 26912 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:46201
I20260809 18:55:57.366004 27012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:46201 every 8 connection(s)
I20260809 18:55:57.366298 26912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:57.366825 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 26912
I20260809 18:55:57.366938 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:57.367095 24427 external_mini_cluster.cc:1428] Setting key 908fe397a967633620c5c3bde36079c8
I20260809 18:55:57.367122 27016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:57.370069 27016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47: Bootstrap starting.
I20260809 18:55:57.370705 27016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:57.371021 27016 log.cc:824] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:57.371753 27016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47: No bootstrap required, opened a new log
I20260809 18:55:57.372211 27020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:57.367523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48761 (local address 127.23.218.254:46201)
0809 18:55:57.367686 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:57.367695 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:57.367717 (+ 22us) server_negotiation.cc:407] Connection header received
0809 18:55:57.368149 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:57.368156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:57.368198 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:57.368307 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:57.368535 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:57.369379 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:57.370443 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:57.370868 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:57.370944 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:57.371589 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:57.371592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:57.371593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:57.371734 (+ 141us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:57.371773 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:57.371783 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:57.371864 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:57.371992 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 18:55:57.372014 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260809 18:55:57.373140 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:57.373793 27016 raft_consensus.cc:348] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afec3d940183447fb3fe10d769be3a47" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46201 } }
I20260809 18:55:57.373920 27016 raft_consensus.cc:374] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:57.373955 27016 raft_consensus.cc:729] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afec3d940183447fb3fe10d769be3a47, State: Initialized, Role: FOLLOWER
I20260809 18:55:57.374089 27016 consensus_queue.cc:260] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afec3d940183447fb3fe10d769be3a47" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46201 } }
I20260809 18:55:57.374155 27016 raft_consensus.cc:388] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:57.374184 27016 raft_consensus.cc:482] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:57.374233 27016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:57.374799 27016 raft_consensus.cc:504] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afec3d940183447fb3fe10d769be3a47" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46201 } }
I20260809 18:55:57.374886 27016 leader_election.cc:302] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afec3d940183447fb3fe10d769be3a47; no voters:
I20260809 18:55:57.375005 27016 leader_election.cc:288] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:57.375053 27032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:57.375203 27016 sys_catalog.cc:564] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:57.375218 27032 raft_consensus.cc:686] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [term 1 LEADER]: Becoming Leader. State: Replica: afec3d940183447fb3fe10d769be3a47, State: Running, Role: LEADER
I20260809 18:55:57.375298 27032 consensus_queue.cc:237] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afec3d940183447fb3fe10d769be3a47" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46201 } }
I20260809 18:55:57.375639 27034 sys_catalog.cc:455] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afec3d940183447fb3fe10d769be3a47. Latest consensus state: current_term: 1 leader_uuid: "afec3d940183447fb3fe10d769be3a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afec3d940183447fb3fe10d769be3a47" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46201 } } }
I20260809 18:55:57.375644 27033 sys_catalog.cc:455] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afec3d940183447fb3fe10d769be3a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afec3d940183447fb3fe10d769be3a47" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 46201 } } }
I20260809 18:55:57.375788 27034 sys_catalog.cc:458] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:57.375809 27033 sys_catalog.cc:458] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:57.376183 27050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:57.376646 27050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:57.378059 27050 catalog_manager.cc:1347] Generated new cluster ID: 11298e94a9134fa1b32e584db58956c1
I20260809 18:55:57.378111 27050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:57.379293 27020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:57.374382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33703 (local address 127.23.218.254:46201)
0809 18:55:57.374415 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:57.374417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:57.376499 (+ 2082us) server_negotiation.cc:407] Connection header received
0809 18:55:57.376567 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:57.376568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:57.376591 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:57.376637 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:57.376992 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:57.377421 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:57.378245 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:55:57.378483 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:55:57.378534 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:55:57.379085 (+ 551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:57.379086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:57.379086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:57.379090 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:57.379104 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:57.379108 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:57.379157 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:57.379219 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 18:55:57.379227 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 18:55:57.382189 27050 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:57.382912 27050 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:57.389529 27050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afec3d940183447fb3fe10d769be3a47: Generated new TSK 0
I20260809 18:55:57.389678 27050 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:57.392351 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 26912
W20260809 18:55:57.413818 26889 connection.cc:535] client connection to 127.23.218.254:46201 recv error: Network error: failed to read from TLS socket (remote: 127.23.218.254:46201): Connection reset by peer (error 104)
2026-08-09T18:55:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:57 dist-test-slave-6zbq krb5kdc[27063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:57 dist-test-slave-6zbq krb5kdc[27063](info): set up 2 sockets
Aug 09 18:55:57 dist-test-slave-6zbq krb5kdc[27063](info): commencing operation
krb5kdc: starting...
W20260809 18:55:58.541553 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:58 dist-test-slave-6zbq krb5kdc[27063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:55:58.648442 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:39875
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:39875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:55:58.725782 27080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:55:58.725968 27080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:55:58.725997 27080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:55:58.727145 27080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:55:58.727187 27080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:55:58.727206 27080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:55:58.727221 27080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:55:58.727237 27080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:55:58.728704 27080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:39875
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:39875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:55:58.728927 27080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:55:58.729121 27080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:55:58.733537 27088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:58.733538 27093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:55:58.733547 27091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:55:58.733769 27080 server_base.cc:1034] running on GCE node
I20260809 18:55:58.733974 27080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:55:58.734210 27080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:55:58.735351 27080 hybrid_clock.cc:648] HybridClock initialized: now 1786301758735333 us; error 30 us; skew 500 ppm
Aug 09 18:55:58 dist-test-slave-6zbq krb5kdc[27063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301758, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:55:58.737537 27080 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:55:58.737918 27080 webserver.cc:460] Webserver started at http://127.23.218.254:35839/ using document root <none> and password file <none>
I20260809 18:55:58.738103 27080 fs_manager.cc:359] Metadata directory not provided
I20260809 18:55:58.738149 27080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:55:58.738266 27080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:55:58.739265 27080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "0ab1f6a6c33149678774ebaff4ba5f41"
format_stamp: "Formatted at 2026-08-09 18:55:58 on dist-test-slave-6zbq"
server_key: "55565dfe2e1f560a78c13c307ac888d9"
server_key_iv: "608cff8ee1b11a86d368172f35328118"
server_key_version: "encryptionkey@0"
I20260809 18:55:58.739593 27080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "0ab1f6a6c33149678774ebaff4ba5f41"
format_stamp: "Formatted at 2026-08-09 18:55:58 on dist-test-slave-6zbq"
server_key: "55565dfe2e1f560a78c13c307ac888d9"
server_key_iv: "608cff8ee1b11a86d368172f35328118"
server_key_version: "encryptionkey@0"
I20260809 18:55:58.740751 27080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 18:55:58.741360 27103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:55:58.741508 27080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:55:58.741603 27080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "0ab1f6a6c33149678774ebaff4ba5f41"
format_stamp: "Formatted at 2026-08-09 18:55:58 on dist-test-slave-6zbq"
server_key: "55565dfe2e1f560a78c13c307ac888d9"
server_key_iv: "608cff8ee1b11a86d368172f35328118"
server_key_version: "encryptionkey@0"
I20260809 18:55:58.741662 27080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:55:58.744721 27080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:55:58.746287 27080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:55:58.746426 27080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:55:58.750002 27080 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:39875
I20260809 18:55:58.750031 27188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:39875 every 8 connection(s)
I20260809 18:55:58.750299 27080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:55:58.751232 27190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:55:58.752135 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27080
I20260809 18:55:58.752259 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:55:58.752414 24427 external_mini_cluster.cc:1428] Setting key 7f7c77d404357c2052eb161a50e2a2f3
I20260809 18:55:58.753762 27190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41: Bootstrap starting.
I20260809 18:55:58.754427 27190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41: Neither blocks nor log segments found. Creating new log.
I20260809 18:55:58.754740 27190 log.cc:824] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41: Log is configured to *not* fsync() on all Append() calls
I20260809 18:55:58.754817 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:58.752860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54175 (local address 127.23.218.254:39875)
0809 18:55:58.753042 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:58.753051 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:58.753083 (+ 32us) server_negotiation.cc:407] Connection header received
0809 18:55:58.753544 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:58.753553 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:58.753586 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:58.753734 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:58.754380 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:58.754384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:58.754384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:58.754389 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:58.754435 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:58.754450 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:58.754535 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:58.754596 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 18:55:58.754636 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260809 18:55:58.755345 27190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41: No bootstrap required, opened a new log
I20260809 18:55:58.755730 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:55:58.757611 27190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab1f6a6c33149678774ebaff4ba5f41" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39875 } }
I20260809 18:55:58.757732 27190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:55:58.757754 27190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab1f6a6c33149678774ebaff4ba5f41, State: Initialized, Role: FOLLOWER
I20260809 18:55:58.757863 27190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab1f6a6c33149678774ebaff4ba5f41" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39875 } }
I20260809 18:55:58.757961 27190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:55:58.757998 27190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:55:58.758045 27190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:55:58.758203 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:55:58.757205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53683 (local address 127.23.218.254:39875)
0809 18:55:58.757242 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:55:58.757244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:55:58.757326 (+ 82us) server_negotiation.cc:407] Connection header received
0809 18:55:58.757365 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:55:58.757365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:55:58.757370 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:55:58.757434 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:55:58.757976 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:55:58.757977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:55:58.757978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:55:58.757982 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:55:58.757997 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:55:58.758001 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:55:58.758051 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 18:55:58.758134 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 18:55:58.758147 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:55:58.758735 27190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab1f6a6c33149678774ebaff4ba5f41" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39875 } }
I20260809 18:55:58.758829 27190 leader_election.cc:302] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ab1f6a6c33149678774ebaff4ba5f41; no voters:
I20260809 18:55:58.758980 27190 leader_election.cc:288] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:55:58.759021 27206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:55:58.759164 27206 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [term 1 LEADER]: Becoming Leader. State: Replica: 0ab1f6a6c33149678774ebaff4ba5f41, State: Running, Role: LEADER
I20260809 18:55:58.759186 27190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:55:58.759253 27206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab1f6a6c33149678774ebaff4ba5f41" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39875 } }
I20260809 18:55:58.759533 27206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ab1f6a6c33149678774ebaff4ba5f41. Latest consensus state: current_term: 1 leader_uuid: "0ab1f6a6c33149678774ebaff4ba5f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab1f6a6c33149678774ebaff4ba5f41" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39875 } } }
I20260809 18:55:58.759651 27206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:58.759559 27208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ab1f6a6c33149678774ebaff4ba5f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ab1f6a6c33149678774ebaff4ba5f41" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 39875 } } }
I20260809 18:55:58.759735 27208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41 [sys.catalog]: This master's current role is: LEADER
I20260809 18:55:58.759991 27219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:55:58.760414 27219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:55:58.761878 27219 catalog_manager.cc:1347] Generated new cluster ID: ee2f95005ff54e26a8095f9299d8bee1
I20260809 18:55:58.761919 27219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:55:58.769902 27219 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:55:58.770066 27219 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:55:58.773293 27219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ab1f6a6c33149678774ebaff4ba5f41: Generated new TSK 0
I20260809 18:55:58.773442 27219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:55:58.776350 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27080
2026-08-09T18:55:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:55:58 dist-test-slave-6zbq krb5kdc[27243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:55:58 dist-test-slave-6zbq krb5kdc[27243](info): set up 2 sockets
Aug 09 18:55:58 dist-test-slave-6zbq krb5kdc[27243](info): commencing operation
krb5kdc: starting...
W20260809 18:55:59.916221 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:55:59 dist-test-slave-6zbq krb5kdc[27243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:55:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:55:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:56:00.022483 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36009
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:36009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:56:00.101504 27262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:56:00.101689 27262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:56:00.101711 27262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:56:00.102806 27262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:56:00.102841 27262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:56:00.102854 27262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:56:00.102864 27262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:56:00.102875 27262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:56:00.104274 27262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:36009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:56:00.104476 27262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:56:00.104637 27262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:56:00.109092 27270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:00.109107 27273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:00.109159 27262 server_base.cc:1034] running on GCE node
W20260809 18:56:00.109107 27277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:00.109556 27262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:56:00.109827 27262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:56:00.110953 27262 hybrid_clock.cc:648] HybridClock initialized: now 1786301760110945 us; error 26 us; skew 500 ppm
Aug 09 18:56:00 dist-test-slave-6zbq krb5kdc[27243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301760, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:56:00.113227 27262 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:56:00.113648 27262 webserver.cc:460] Webserver started at http://127.23.218.254:38431/ using document root <none> and password file <none>
I20260809 18:56:00.113865 27262 fs_manager.cc:359] Metadata directory not provided
I20260809 18:56:00.113914 27262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:56:00.114035 27262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:56:00.115084 27262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "93488fb713554cb39aaa4a7d089707eb"
format_stamp: "Formatted at 2026-08-09 18:56:00 on dist-test-slave-6zbq"
server_key: "7ad566ae93039c64fc04cebb25ecf445"
server_key_iv: "0baa50253d21f0429f3a60d709c51295"
server_key_version: "encryptionkey@0"
I20260809 18:56:00.115412 27262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "93488fb713554cb39aaa4a7d089707eb"
format_stamp: "Formatted at 2026-08-09 18:56:00 on dist-test-slave-6zbq"
server_key: "7ad566ae93039c64fc04cebb25ecf445"
server_key_iv: "0baa50253d21f0429f3a60d709c51295"
server_key_version: "encryptionkey@0"
I20260809 18:56:00.116595 27262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:56:00.117228 27289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:00.117377 27262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:56:00.117460 27262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "93488fb713554cb39aaa4a7d089707eb"
format_stamp: "Formatted at 2026-08-09 18:56:00 on dist-test-slave-6zbq"
server_key: "7ad566ae93039c64fc04cebb25ecf445"
server_key_iv: "0baa50253d21f0429f3a60d709c51295"
server_key_version: "encryptionkey@0"
I20260809 18:56:00.117519 27262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:56:00.125550 27262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:56:00.127346 27262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:56:00.127477 27262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:56:00.131313 27262 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:36009
I20260809 18:56:00.131345 27373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:36009 every 8 connection(s)
I20260809 18:56:00.131659 27262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:56:00.132508 27374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:56:00.134862 27374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb: Bootstrap starting.
I20260809 18:56:00.135429 27374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb: Neither blocks nor log segments found. Creating new log.
I20260809 18:56:00.135681 27374 log.cc:824] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb: Log is configured to *not* fsync() on all Append() calls
I20260809 18:56:00.136363 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27262
I20260809 18:56:00.136395 27374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb: No bootstrap required, opened a new log
I20260809 18:56:00.136482 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:56:00.136679 24427 external_mini_cluster.cc:1428] Setting key 50ff4c84b929b64ed62ee4910fc6de6f
I20260809 18:56:00.138252 27374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93488fb713554cb39aaa4a7d089707eb" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36009 } }
I20260809 18:56:00.138440 27374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:56:00.138521 27374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93488fb713554cb39aaa4a7d089707eb, State: Initialized, Role: FOLLOWER
I20260809 18:56:00.138665 27374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93488fb713554cb39aaa4a7d089707eb" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36009 } }
I20260809 18:56:00.138773 27374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:56:00.138813 27374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:56:00.138862 27374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:56:00.139760 27374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93488fb713554cb39aaa4a7d089707eb" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36009 } }
I20260809 18:56:00.139853 27374 leader_election.cc:302] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93488fb713554cb39aaa4a7d089707eb; no voters:
I20260809 18:56:00.140005 27374 leader_election.cc:288] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:56:00.140048 27383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:56:00.140249 27383 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [term 1 LEADER]: Becoming Leader. State: Replica: 93488fb713554cb39aaa4a7d089707eb, State: Running, Role: LEADER
I20260809 18:56:00.140254 27374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:56:00.140406 27383 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93488fb713554cb39aaa4a7d089707eb" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36009 } }
I20260809 18:56:00.141644 27384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93488fb713554cb39aaa4a7d089707eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93488fb713554cb39aaa4a7d089707eb" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36009 } } }
I20260809 18:56:00.141728 27384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:00.141861 27386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93488fb713554cb39aaa4a7d089707eb. Latest consensus state: current_term: 1 leader_uuid: "93488fb713554cb39aaa4a7d089707eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93488fb713554cb39aaa4a7d089707eb" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36009 } } }
I20260809 18:56:00.142022 27386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb [sys.catalog]: This master's current role is: LEADER
W20260809 18:56:00.142222 27409 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:56:00.142274 27409 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:56:00.142333 27410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:56:00.142514 27410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:56:00.143903 27410 catalog_manager.cc:1347] Generated new cluster ID: 03a79da253a046a2ae59e96a306e23b9
I20260809 18:56:00.143953 27410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 18:56:00 dist-test-slave-6zbq krb5kdc[27243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:56:00.148615 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:00.137190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60417 (local address 127.23.218.254:36009)
0809 18:56:00.137360 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:00.137368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:00.137393 (+ 25us) server_negotiation.cc:407] Connection header received
0809 18:56:00.138222 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:00.138228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:00.138280 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:00.138419 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:00.139193 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:00.140209 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:00.141365 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:00.141975 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:00.142082 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:00.144904 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:00.144908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:00.144908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:00.144920 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:00.146724 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:00.147079 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:00.147082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:00.147082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:00.147199 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:00.147383 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:00.147384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:00.147384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:00.147666 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:00.147801 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:00.148267 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 18:56:00.148403 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260809 18:56:00.149518 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:56:00.151144 27410 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:56:00.151633 27410 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:56:00.156590 27410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93488fb713554cb39aaa4a7d089707eb: Generated new TSK 0
I20260809 18:56:00.156749 27410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:56:00.161128 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:00.150948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42011 (local address 127.23.218.254:36009)
0809 18:56:00.150980 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:00.150982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:00.151127 (+ 145us) server_negotiation.cc:407] Connection header received
0809 18:56:00.154497 (+ 3370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:00.154498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:00.154510 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:00.154556 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:00.155177 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:00.155923 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:00.156948 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:00.157160 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:00.157214 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:00.158462 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:00.158463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:00.158463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:00.158470 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:00.160172 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:00.160530 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:00.160531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:00.160531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:00.160575 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:00.160745 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:00.160745 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:00.160745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:00.160828 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:00.160903 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:00.161005 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 18:56:00.161061 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 18:56:00.162825 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27262
2026-08-09T18:56:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:56:00 dist-test-slave-6zbq krb5kdc[27430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:56:00 dist-test-slave-6zbq krb5kdc[27430](info): set up 2 sockets
Aug 09 18:56:00 dist-test-slave-6zbq krb5kdc[27430](info): commencing operation
krb5kdc: starting...
W20260809 18:56:01.293212 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:56:01 dist-test-slave-6zbq krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:56:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:56:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:56:01.397740 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38757
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:33435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:38757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:56:01.476357 27449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:56:01.476558 27449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:56:01.476585 27449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:56:01.477659 27449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:56:01.477700 27449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:56:01.477715 27449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:56:01.477725 27449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:56:01.477741 27449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:56:01.479144 27449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:33435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:38757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:38757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:56:01.479329 27449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:56:01.479491 27449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:56:01.483986 27466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:01.484084 27449 server_base.cc:1034] running on GCE node
W20260809 18:56:01.483999 27458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:01.483999 27460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:01.484454 27449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:56:01.484712 27449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:56:01.485867 27449 hybrid_clock.cc:648] HybridClock initialized: now 1786301761485852 us; error 28 us; skew 500 ppm
Aug 09 18:56:01 dist-test-slave-6zbq krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301761, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:56:01.491236 27449 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:56:01.491621 27449 webserver.cc:460] Webserver started at http://127.23.218.254:46577/ using document root <none> and password file <none>
I20260809 18:56:01.491820 27449 fs_manager.cc:359] Metadata directory not provided
I20260809 18:56:01.491868 27449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:56:01.491987 27449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:56:01.492950 27449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "f866c6e076a344fa8612e62f27baa0f5"
format_stamp: "Formatted at 2026-08-09 18:56:01 on dist-test-slave-6zbq"
server_key: "261f461f1b14381cf891b40399eeec2f"
server_key_iv: "35bbdcecec66e50509c6fda5a26a32c1"
server_key_version: "encryptionkey@0"
I20260809 18:56:01.493273 27449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "f866c6e076a344fa8612e62f27baa0f5"
format_stamp: "Formatted at 2026-08-09 18:56:01 on dist-test-slave-6zbq"
server_key: "261f461f1b14381cf891b40399eeec2f"
server_key_iv: "35bbdcecec66e50509c6fda5a26a32c1"
server_key_version: "encryptionkey@0"
I20260809 18:56:01.494494 27449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:56:01.495136 27477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:01.495273 27449 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 18:56:01.495357 27449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "f866c6e076a344fa8612e62f27baa0f5"
format_stamp: "Formatted at 2026-08-09 18:56:01 on dist-test-slave-6zbq"
server_key: "261f461f1b14381cf891b40399eeec2f"
server_key_iv: "35bbdcecec66e50509c6fda5a26a32c1"
server_key_version: "encryptionkey@0"
I20260809 18:56:01.495422 27449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:56:01.504782 27449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:56:01.506294 27449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:56:01.506436 27449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:56:01.509986 27449 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:38757
I20260809 18:56:01.510015 27566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:38757 every 8 connection(s)
I20260809 18:56:01.510296 27449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:56:01.511118 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:56:01.511415 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27449
I20260809 18:56:01.511524 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:56:01.511672 24427 external_mini_cluster.cc:1428] Setting key 0c356c35313e1236d2bb9e29b3c4c605
I20260809 18:56:01.513880 27567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5: Bootstrap starting.
I20260809 18:56:01.514546 27567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5: Neither blocks nor log segments found. Creating new log.
I20260809 18:56:01.514815 27567 log.cc:824] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5: Log is configured to *not* fsync() on all Append() calls
I20260809 18:56:01.515450 27567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5: No bootstrap required, opened a new log
I20260809 18:56:01.517056 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:01.512117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44677 (local address 127.23.218.254:38757)
0809 18:56:01.512296 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:01.512304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:01.512337 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:56:01.512812 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:01.512821 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:01.512879 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:01.513027 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:01.513183 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:01.514475 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:01.515228 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:01.515784 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:01.515892 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:01.516447 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:01.516452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:01.516452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:56:01.516630 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:01.516668 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:56:01.516683 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:01.516749 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:01.516798 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 18:56:01.516860 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260809 18:56:01.517724 27567 raft_consensus.cc:348] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f866c6e076a344fa8612e62f27baa0f5" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38757 } }
I20260809 18:56:01.517864 27567 raft_consensus.cc:374] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:56:01.517886 27567 raft_consensus.cc:729] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f866c6e076a344fa8612e62f27baa0f5, State: Initialized, Role: FOLLOWER
I20260809 18:56:01.518003 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:56:01.518002 27567 consensus_queue.cc:260] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f866c6e076a344fa8612e62f27baa0f5" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38757 } }
I20260809 18:56:01.518066 27567 raft_consensus.cc:388] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:56:01.518103 27567 raft_consensus.cc:482] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:56:01.518148 27567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:56:01.518677 27567 raft_consensus.cc:504] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f866c6e076a344fa8612e62f27baa0f5" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38757 } }
I20260809 18:56:01.518765 27567 leader_election.cc:302] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f866c6e076a344fa8612e62f27baa0f5; no voters:
I20260809 18:56:01.518921 27567 leader_election.cc:288] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:56:01.518970 27574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:56:01.519142 27567 sys_catalog.cc:564] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:56:01.519266 27574 raft_consensus.cc:686] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [term 1 LEADER]: Becoming Leader. State: Replica: f866c6e076a344fa8612e62f27baa0f5, State: Running, Role: LEADER
I20260809 18:56:01.519448 27574 consensus_queue.cc:237] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f866c6e076a344fa8612e62f27baa0f5" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38757 } }
I20260809 18:56:01.519824 27581 sys_catalog.cc:455] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f866c6e076a344fa8612e62f27baa0f5. Latest consensus state: current_term: 1 leader_uuid: "f866c6e076a344fa8612e62f27baa0f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f866c6e076a344fa8612e62f27baa0f5" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38757 } } }
I20260809 18:56:01.519901 27581 sys_catalog.cc:458] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:01.520076 27578 sys_catalog.cc:455] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f866c6e076a344fa8612e62f27baa0f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f866c6e076a344fa8612e62f27baa0f5" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 38757 } } }
I20260809 18:56:01.520200 27578 sys_catalog.cc:458] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5 [sys.catalog]: This master's current role is: LEADER
W20260809 18:56:01.520804 27601 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:56:01.520859 27601 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:56:01.520900 27590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:56:01.521102 27590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:56:01.522303 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:01.519319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60387 (local address 127.23.218.254:38757)
0809 18:56:01.519349 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:01.519351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:01.519536 (+ 185us) server_negotiation.cc:407] Connection header received
0809 18:56:01.519623 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:01.519624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:01.519635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:01.519663 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:01.519867 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:01.520496 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:01.521304 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:01.521508 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:01.521561 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:01.522106 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:01.522107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:01.522107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:56:01.522111 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:01.522123 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:56:01.522127 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:01.522176 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:01.522219 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 18:56:01.522239 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 18:56:01.522544 27590 catalog_manager.cc:1347] Generated new cluster ID: bfd1c9ecc9f3402496dca47cb35dd1ed
I20260809 18:56:01.522598 27590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:56:01.531462 27590 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:56:01.531932 27590 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:56:01.537065 27590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f866c6e076a344fa8612e62f27baa0f5: Generated new TSK 0
I20260809 18:56:01.537207 27590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:56:01.542901 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27449
2026-08-09T18:56:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:56:01 dist-test-slave-6zbq krb5kdc[27608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:56:01 dist-test-slave-6zbq krb5kdc[27608](info): set up 2 sockets
Aug 09 18:56:01 dist-test-slave-6zbq krb5kdc[27608](info): commencing operation
krb5kdc: starting...
W20260809 18:56:02.688412 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:56:02 dist-test-slave-6zbq krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:56:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:56:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:56:02.792124 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:35519
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:41621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:35519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:56:02.869601 27630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:56:02.869781 27630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:56:02.869802 27630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:56:02.870884 27630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:56:02.870918 27630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:56:02.870931 27630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:56:02.870941 27630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:56:02.870952 27630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:56:02.872352 27630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:41621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:35519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:35519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:56:02.872530 27630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:56:02.872687 27630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:56:02.877250 27651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:02.877355 27630 server_base.cc:1034] running on GCE node
W20260809 18:56:02.877260 27653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:02.877305 27647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:02.877655 27630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:56:02.877905 27630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:56:02.879036 27630 hybrid_clock.cc:648] HybridClock initialized: now 1786301762879015 us; error 34 us; skew 500 ppm
Aug 09 18:56:02 dist-test-slave-6zbq krb5kdc[27608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301762, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:56:02.881263 27630 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:56:02.881671 27630 webserver.cc:460] Webserver started at http://127.23.218.254:40281/ using document root <none> and password file <none>
I20260809 18:56:02.881876 27630 fs_manager.cc:359] Metadata directory not provided
I20260809 18:56:02.881922 27630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:56:02.882040 27630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:56:02.883030 27630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "57bfb1e13b28474bab42a03655af76ec"
format_stamp: "Formatted at 2026-08-09 18:56:02 on dist-test-slave-6zbq"
server_key: "ee4f043294c5ba0d772cf4def57016d0"
server_key_iv: "146676806be437d8a90a3ea408ff34e4"
server_key_version: "encryptionkey@0"
I20260809 18:56:02.883347 27630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "57bfb1e13b28474bab42a03655af76ec"
format_stamp: "Formatted at 2026-08-09 18:56:02 on dist-test-slave-6zbq"
server_key: "ee4f043294c5ba0d772cf4def57016d0"
server_key_iv: "146676806be437d8a90a3ea408ff34e4"
server_key_version: "encryptionkey@0"
I20260809 18:56:02.884507 27630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:56:02.885139 27665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:02.885278 27630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:56:02.885349 27630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "57bfb1e13b28474bab42a03655af76ec"
format_stamp: "Formatted at 2026-08-09 18:56:02 on dist-test-slave-6zbq"
server_key: "ee4f043294c5ba0d772cf4def57016d0"
server_key_iv: "146676806be437d8a90a3ea408ff34e4"
server_key_version: "encryptionkey@0"
I20260809 18:56:02.885409 27630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:56:02.888537 27630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:56:02.890105 27630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:56:02.890233 27630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:56:02.893911 27630 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:35519
I20260809 18:56:02.893939 27751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:35519 every 8 connection(s)
I20260809 18:56:02.894212 27630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:56:02.895102 27757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:56:02.895833 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27630
I20260809 18:56:02.895951 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:56:02.896130 24427 external_mini_cluster.cc:1428] Setting key c4652e18beef90275d06def4df5a3cfa
I20260809 18:56:02.897428 27757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec: Bootstrap starting.
I20260809 18:56:02.897904 27757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec: Neither blocks nor log segments found. Creating new log.
I20260809 18:56:02.898131 27757 log.cc:824] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec: Log is configured to *not* fsync() on all Append() calls
I20260809 18:56:02.898737 27757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec: No bootstrap required, opened a new log
I20260809 18:56:02.900096 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:02.896589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60965 (local address 127.23.218.254:35519)
0809 18:56:02.896758 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:02.896765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:02.898470 (+ 1705us) server_negotiation.cc:407] Connection header received
0809 18:56:02.898937 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:02.898944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:02.898969 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:02.899093 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:02.899685 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:02.899689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:02.899689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:56:02.899693 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:02.899733 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:56:02.899743 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:02.899812 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:02.899877 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 18:56:02.899914 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260809 18:56:02.900817 27757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfb1e13b28474bab42a03655af76ec" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35519 } }
I20260809 18:56:02.900976 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:56:02.900970 27757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 18:56:02.901015 27757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57bfb1e13b28474bab42a03655af76ec, State: Initialized, Role: FOLLOWER
I20260809 18:56:02.901140 27757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfb1e13b28474bab42a03655af76ec" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35519 } }
I20260809 18:56:02.901223 27757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 18:56:02.901257 27757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 18:56:02.901298 27757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:56:02.901896 27757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfb1e13b28474bab42a03655af76ec" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35519 } }
I20260809 18:56:02.902011 27757 leader_election.cc:302] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57bfb1e13b28474bab42a03655af76ec; no voters:
I20260809 18:56:02.902159 27757 leader_election.cc:288] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 18:56:02.902189 27768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:56:02.902443 27757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:56:02.902608 27768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [term 1 LEADER]: Becoming Leader. State: Replica: 57bfb1e13b28474bab42a03655af76ec, State: Running, Role: LEADER
I20260809 18:56:02.902761 27768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfb1e13b28474bab42a03655af76ec" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35519 } }
I20260809 18:56:02.903101 27776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57bfb1e13b28474bab42a03655af76ec. Latest consensus state: current_term: 1 leader_uuid: "57bfb1e13b28474bab42a03655af76ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfb1e13b28474bab42a03655af76ec" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35519 } } }
I20260809 18:56:02.903095 27771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57bfb1e13b28474bab42a03655af76ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfb1e13b28474bab42a03655af76ec" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 35519 } } }
I20260809 18:56:02.903183 27776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:02.903189 27771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:02.903501 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:02.902438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54037 (local address 127.23.218.254:35519)
0809 18:56:02.902468 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:02.902470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:02.902612 (+ 142us) server_negotiation.cc:407] Connection header received
0809 18:56:02.902700 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:02.902701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:02.902705 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:02.902731 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:02.903313 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:02.903315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:02.903315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 18:56:02.903319 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:02.903332 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 18:56:02.903336 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:02.903389 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:02.903439 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 18:56:02.903451 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 18:56:02.904243 27793 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:56:02.904290 27793 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:56:02.904350 27787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:56:02.904532 27787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:56:02.905799 27787 catalog_manager.cc:1347] Generated new cluster ID: 151aaae2eac543b0b760265d1695e83b
I20260809 18:56:02.905848 27787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:56:02.917035 27787 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:56:02.917165 27787 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:56:02.923530 27787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57bfb1e13b28474bab42a03655af76ec: Generated new TSK 0
I20260809 18:56:02.923671 27787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 18:56:02.926131 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27630
2026-08-09T18:56:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1373 ms)
[----------] 18 tests from AuthTokenIssuingTest (25126 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:56:02 dist-test-slave-6zbq krb5kdc[27806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:56:02 dist-test-slave-6zbq krb5kdc[27806](info): set up 2 sockets
Aug 09 18:56:02 dist-test-slave-6zbq krb5kdc[27806](info): commencing operation
krb5kdc: starting...
W20260809 18:56:04.072441 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:56:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:56:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:56:04.176002 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:33255
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:33255,127.23.218.253:36965,127.23.218.252:43253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:56:04.255678 27825 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 18:56:04.255857 27825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:56:04.255882 27825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:56:04.255908 27825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:56:04.257012 27825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:56:04.257050 27825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:56:04.257069 27825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:56:04.257085 27825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:56:04.257117 27825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:56:04.258561 27825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:33255,127.23.218.253:36965,127.23.218.252:43253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:33255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:56:04.258767 27825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:56:04.258952 27825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:56:04.263204 27840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:04.263214 27835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:04.263382 27825 server_base.cc:1034] running on GCE node
W20260809 18:56:04.263238 27838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:04.263650 27825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:56:04.263922 27825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:56:04.265061 27825 hybrid_clock.cc:648] HybridClock initialized: now 1786301764265048 us; error 25 us; skew 500 ppm
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:56:04.267310 27825 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:56:04.267689 27825 webserver.cc:460] Webserver started at http://127.23.218.254:45505/ using document root <none> and password file <none>
I20260809 18:56:04.267897 27825 fs_manager.cc:359] Metadata directory not provided
I20260809 18:56:04.267943 27825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:56:04.268059 27825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:56:04.268996 27825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "78ae6826c04e44f6b86e375edc481e79"
format_stamp: "Formatted at 2026-08-09 18:56:04 on dist-test-slave-6zbq"
server_key: "b4e67ed0176cf557960fafac562cb10e"
server_key_iv: "bf29e829b2710b04fbbb6180a641565a"
server_key_version: "encryptionkey@0"
I20260809 18:56:04.269325 27825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "78ae6826c04e44f6b86e375edc481e79"
format_stamp: "Formatted at 2026-08-09 18:56:04 on dist-test-slave-6zbq"
server_key: "b4e67ed0176cf557960fafac562cb10e"
server_key_iv: "bf29e829b2710b04fbbb6180a641565a"
server_key_version: "encryptionkey@0"
I20260809 18:56:04.270510 27825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 18:56:04.271126 27856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:04.271275 27825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:04.271353 27825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "78ae6826c04e44f6b86e375edc481e79"
format_stamp: "Formatted at 2026-08-09 18:56:04 on dist-test-slave-6zbq"
server_key: "b4e67ed0176cf557960fafac562cb10e"
server_key_iv: "bf29e829b2710b04fbbb6180a641565a"
server_key_version: "encryptionkey@0"
I20260809 18:56:04.271416 27825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:56:04.280288 27825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:56:04.282130 27825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:56:04.282251 27825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:56:04.285825 27825 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:33255
I20260809 18:56:04.285856 27933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:33255 every 8 connection(s)
I20260809 18:56:04.286126 27825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:56:04.286885 27934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:56:04.288236 27934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:04.289558 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27825
I20260809 18:56:04.289662 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:56:04.289825 24427 external_mini_cluster.cc:1428] Setting key 9ecc54fa3d46df7dbc2585867c069b24
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:56:04.296917 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.288805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51798 (local address 127.23.218.254:33255)
0809 18:56:04.288939 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.288948 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.289050 (+ 102us) server_negotiation.cc:407] Connection header received
0809 18:56:04.289555 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.289557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.289573 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.289647 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.290186 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.291011 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.291796 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.292079 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.292166 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.293957 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.293964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.293964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.293970 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.295525 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.295945 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.295947 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.295948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.296016 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.296231 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.296232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.296232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.296401 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.296456 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.296660 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.296762 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260809 18:56:04.296950 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.288764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:33255 (local address 127.0.0.1:51798)
0809 18:56:04.288931 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.288970 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:04.289421 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:04.289699 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:04.289702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:04.289949 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:04.290098 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.290102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.291094 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.291095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.291722 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.291724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.292148 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.292149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.292224 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.292618 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:04.292627 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:04.293869 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:04.295609 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.295616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.295618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.295874 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.296072 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.296073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.296073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.296174 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.296463 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:04.296469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:04.296539 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 18:56:04.296635 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 18:56:04.296822 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":50,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.23.218.253@KRBTEST.COM; defaulting to no policy
I20260809 18:56:04.297798 27934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:04.298067 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.297963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:36965 (local address 127.0.0.1:53362)
0809 18:56:04.298007 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.298048 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 18:56:04.298182 27858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
I20260809 18:56:04.298584 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.298498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:36965 (local address 127.0.0.1:53376)
0809 18:56:04.298540 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.298566 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 18:56:04.298678 27934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.253:36965: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab.
Entry for principal kudu/127.23.218.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab.
I20260809 18:56:04.311158 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.253:36965
--webserver_interface=127.23.218.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab
--principal=kudu/127.23.218.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:33255,127.23.218.253:36965,127.23.218.252:43253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:56:04.347885 27934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } attempt: 1
I20260809 18:56:04.348255 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.348134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:36965 (local address 127.0.0.1:53388)
0809 18:56:04.348194 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.348229 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 18:56:04.348536 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.348462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:36965 (local address 127.0.0.1:53392)
0809 18:56:04.348496 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.348519 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 18:56:04.348621 27934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.253:36965: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
W20260809 18:56:04.388236 27951 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 18:56:04.388407 27951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:56:04.388425 27951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:56:04.388458 27951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:56:04.389570 27951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:56:04.389611 27951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:56:04.389624 27951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:56:04.389647 27951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:56:04.389670 27951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:56:04.391103 27951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:33255,127.23.218.253:36965,127.23.218.252:43253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab
--principal=kudu/127.23.218.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.253:36965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.218.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:56:04.391309 27951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:56:04.391489 27951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:56:04.395793 27962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:04.395848 27951 server_base.cc:1034] running on GCE node
W20260809 18:56:04.395784 27959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:04.395785 27960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:04.396232 27951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:56:04.396454 27951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:56:04.397580 27951 hybrid_clock.cc:648] HybridClock initialized: now 1786301764397563 us; error 34 us; skew 500 ppm
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:56:04.399761 27951 init.cc:377] Logged in from keytab as kudu/127.23.218.253@KRBTEST.COM (short username kudu)
I20260809 18:56:04.400118 27951 webserver.cc:460] Webserver started at http://127.23.218.253:46313/ using document root <none> and password file <none>
I20260809 18:56:04.400300 27951 fs_manager.cc:359] Metadata directory not provided
I20260809 18:56:04.400341 27951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:56:04.400460 27951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:56:04.401391 27951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/data/instance:
uuid: "2a8b1aa291444e8dba62cbe78cf6467c"
format_stamp: "Formatted at 2026-08-09 18:56:04 on dist-test-slave-6zbq"
server_key: "a9c4e4e61da05203b8f827fb2399dbd3"
server_key_iv: "17e666bdef318724f6d47cc97744806f"
server_key_version: "encryptionkey@0"
I20260809 18:56:04.401690 27951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/wal/instance:
uuid: "2a8b1aa291444e8dba62cbe78cf6467c"
format_stamp: "Formatted at 2026-08-09 18:56:04 on dist-test-slave-6zbq"
server_key: "a9c4e4e61da05203b8f827fb2399dbd3"
server_key_iv: "17e666bdef318724f6d47cc97744806f"
server_key_version: "encryptionkey@0"
I20260809 18:56:04.402869 27951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 18:56:04.403481 27972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:04.403620 27951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:04.403695 27951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/wal
uuid: "2a8b1aa291444e8dba62cbe78cf6467c"
format_stamp: "Formatted at 2026-08-09 18:56:04 on dist-test-slave-6zbq"
server_key: "a9c4e4e61da05203b8f827fb2399dbd3"
server_key_iv: "17e666bdef318724f6d47cc97744806f"
server_key_version: "encryptionkey@0"
I20260809 18:56:04.403757 27951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:56:04.416769 27934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } attempt: 2
I20260809 18:56:04.417084 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.416981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:36965 (local address 127.0.0.1:53402)
0809 18:56:04.417028 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.417057 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 18:56:04.417336 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.417271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:36965 (local address 127.0.0.1:53418)
0809 18:56:04.417300 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.417322 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 18:56:04.417410 27934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.253:36965: Network error: Client connection negotiation failed: client connection to 127.23.218.253:36965: connect: Connection refused (error 111)
I20260809 18:56:04.425084 27951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:56:04.426921 27951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:56:04.427021 27951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:56:04.430609 27951 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.253:36965
I20260809 18:56:04.430640 28052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.253:36965 every 8 connection(s)
I20260809 18:56:04.430903 27951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
I20260809 18:56:04.431614 28056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:56:04.432933 28056 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:04.434985 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 27951
I20260809 18:56:04.435091 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-1/wal/instance
I20260809 18:56:04.435285 24427 external_mini_cluster.cc:1428] Setting key 83eececc378a782992d20dd109b3f1f9
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:56:04.441416 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.433481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51800 (local address 127.23.218.254:33255)
0809 18:56:04.433534 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.433536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.433680 (+ 144us) server_negotiation.cc:407] Connection header received
0809 18:56:04.434097 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.434098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.434109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.434147 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.434704 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.435378 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.436281 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.436559 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.436624 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.438762 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.438763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.438764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.438771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.440169 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.440560 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.440561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.440562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.440622 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.440828 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.440828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.440829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.440960 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.441029 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.441241 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.441329 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 18:56:04.441444 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.433444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:33255 (local address 127.0.0.1:51800)
0809 18:56:04.433578 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.433614 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:04.434013 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:04.434201 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:04.434204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:04.434447 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:04.434634 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.434639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.435470 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.435471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.436200 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.436202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.436646 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.436646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.436779 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.437186 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:04.437195 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:04.438671 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:04.440268 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.440275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.440277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.440486 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.440682 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.440683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.440683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.440770 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.441017 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:04.441019 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:04.441126 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 18:56:04.441200 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 18:56:04.441308 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260809 18:56:04.442025 28056 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.218.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.252@KRBTEST.COM" created.
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
I20260809 18:56:04.448549 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.442244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:36965 (local address 127.0.0.1:53432)
0809 18:56:04.442281 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.442290 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:04.442328 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:04.442597 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:04.442598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:04.442730 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:04.442786 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.442788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.443584 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.443585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.444050 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.444052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.444466 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.444467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.444527 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.444850 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:04.444855 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:04.446235 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:04.447658 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.447659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.447660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.447814 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.447983 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.447984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.447984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.448036 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.448266 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:04.448267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:04.448302 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 18:56:04.448362 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 18:56:04.448453 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 18:56:04.448618 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.442310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.23.218.253:36965)
0809 18:56:04.442465 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.442468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.442485 (+ 17us) server_negotiation.cc:407] Connection header received
0809 18:56:04.442501 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.442502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.442512 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.442575 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.442855 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.443498 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.444123 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.444403 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.444459 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.446312 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.446320 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.446320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.446327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.447587 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.447879 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.447882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.447883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.447931 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.448085 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.448085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.448086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.448208 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.448284 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.448403 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.448504 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":130,"thread_start_us":106,"threads_started":1}
I20260809 18:56:04.449285 28056 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:04.449563 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.449460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56244)
0809 18:56:04.449498 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.449541 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 18:56:04.449668 27975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
I20260809 18:56:04.450035 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.449953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56258)
0809 18:56:04.449989 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.450013 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 18:56:04.450160 28056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:43253: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab.
Entry for principal kudu/127.23.218.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab.
I20260809 18:56:04.458389 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.252:43253
--webserver_interface=127.23.218.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab
--principal=kudu/127.23.218.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:39675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:33255,127.23.218.253:36965,127.23.218.252:43253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:56:04.499375 28056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } attempt: 1
I20260809 18:56:04.499778 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.499653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56264)
0809 18:56:04.499707 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.499748 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 18:56:04.500051 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.499974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56276)
0809 18:56:04.500010 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.500034 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 18:56:04.500138 28056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:43253: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
I20260809 18:56:04.508550 27934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } attempt: 3
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
I20260809 18:56:04.515259 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.508814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:36965 (local address 127.0.0.1:53436)
0809 18:56:04.508871 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.508882 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:04.508930 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:04.509052 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:04.509052 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:04.509183 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:04.509243 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.509244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.510133 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.510133 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.510671 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.510673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.511134 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.511134 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.511181 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.511613 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:04.511619 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:04.512901 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:04.514461 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.514462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.514462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.514616 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.514786 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.514786 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.514787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.514832 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.515055 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:04.515055 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:04.515089 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 18:56:04.515132 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 18:56:04.515196 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 18:56:04.515314 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.508859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.23.218.253:36965)
0809 18:56:04.508916 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.508918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.508929 (+ 11us) server_negotiation.cc:407] Connection header received
0809 18:56:04.508974 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.508975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.508988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.509046 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.509310 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.510064 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.510731 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.511073 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.511140 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.512982 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.512984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.512984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.512991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.514394 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.514676 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.514677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.514678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.514736 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.514881 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.514882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.514882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.515002 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.515068 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.515156 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.515233 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 18:56:04.515658 27934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:04.515954 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.515855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56288)
0809 18:56:04.515909 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.515933 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 18:56:04.516284 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.516209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56304)
0809 18:56:04.516246 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.516269 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 18:56:04.516376 27934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:43253: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
W20260809 18:56:04.538823 28074 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 18:56:04.538992 28074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:56:04.539008 28074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:56:04.539041 28074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:56:04.540167 28074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:56:04.540203 28074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:56:04.540216 28074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:56:04.540239 28074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:56:04.540261 28074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:56:04.541713 28074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:39675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:33255,127.23.218.253:36965,127.23.218.252:43253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab
--principal=kudu/127.23.218.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.252:43253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.218.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:56:04.541929 28074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:56:04.542122 28074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:56:04.546367 28084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:04.546324 28082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:04.546523 28074 server_base.cc:1034] running on GCE node
W20260809 18:56:04.546379 28086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:04.546789 28074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:56:04.547034 28074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:56:04.547490 27934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } attempt: 1
I20260809 18:56:04.547776 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.547681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56320)
0809 18:56:04.547720 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.547744 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 18:56:04.548020 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.547946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56322)
0809 18:56:04.547980 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.548000 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 18:56:04.548094 27934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:43253: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
I20260809 18:56:04.548178 28074 hybrid_clock.cc:648] HybridClock initialized: now 1786301764548166 us; error 22 us; skew 500 ppm
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:56:04.550446 28074 init.cc:377] Logged in from keytab as kudu/127.23.218.252@KRBTEST.COM (short username kudu)
I20260809 18:56:04.550822 28074 webserver.cc:460] Webserver started at http://127.23.218.252:35487/ using document root <none> and password file <none>
I20260809 18:56:04.551021 28074 fs_manager.cc:359] Metadata directory not provided
I20260809 18:56:04.551061 28074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:56:04.551177 28074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:56:04.552124 28074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/data/instance:
uuid: "c26713c9cef240e7a1210865a7090253"
format_stamp: "Formatted at 2026-08-09 18:56:04 on dist-test-slave-6zbq"
server_key: "8624ef79471a3688c9f155ad0895f3fa"
server_key_iv: "e03cfee9230c079a6362a8482ff5f647"
server_key_version: "encryptionkey@0"
I20260809 18:56:04.552431 28074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/wal/instance:
uuid: "c26713c9cef240e7a1210865a7090253"
format_stamp: "Formatted at 2026-08-09 18:56:04 on dist-test-slave-6zbq"
server_key: "8624ef79471a3688c9f155ad0895f3fa"
server_key_iv: "e03cfee9230c079a6362a8482ff5f647"
server_key_version: "encryptionkey@0"
I20260809 18:56:04.553529 28074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 18:56:04.554111 28096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:04.554244 28074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:04.554314 28074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/wal
uuid: "c26713c9cef240e7a1210865a7090253"
format_stamp: "Formatted at 2026-08-09 18:56:04 on dist-test-slave-6zbq"
server_key: "8624ef79471a3688c9f155ad0895f3fa"
server_key_iv: "e03cfee9230c079a6362a8482ff5f647"
server_key_version: "encryptionkey@0"
I20260809 18:56:04.554396 28074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:56:04.564114 28074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:56:04.565924 28074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:56:04.566043 28074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:56:04.568288 28056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } attempt: 2
I20260809 18:56:04.568619 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.568513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56328)
0809 18:56:04.568549 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.568576 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 18:56:04.568944 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.568828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56330)
0809 18:56:04.568882 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.568918 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 18:56:04.569047 28056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:43253: Network error: Client connection negotiation failed: client connection to 127.23.218.252:43253: connect: Connection refused (error 111)
I20260809 18:56:04.569984 28074 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.252:43253
I20260809 18:56:04.570015 28181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.252:43253 every 8 connection(s)
I20260809 18:56:04.570297 28074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
I20260809 18:56:04.571094 28188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:56:04.572346 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 28074
I20260809 18:56:04.572472 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786301499543878-24427-0/minicluster-data/master-2/wal/instance
I20260809 18:56:04.572664 24427 external_mini_cluster.cc:1428] Setting key ac0ec5536d301ca2e3db7f8722bfd9d0
I20260809 18:56:04.572465 28188 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
I20260809 18:56:04.581048 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.573190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.23.218.254:33255)
0809 18:56:04.573229 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.573230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.573393 (+ 163us) server_negotiation.cc:407] Connection header received
0809 18:56:04.573780 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.573781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.573792 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.573841 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.574535 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.575300 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.576397 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.576602 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.576665 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.578682 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.578683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.578684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.578691 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.579844 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.580246 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.580247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.580248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.580300 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.580506 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.580507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.580507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.580633 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.580703 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.580911 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.580970 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 18:56:04.581202 28195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.573087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:33255 (local address 127.0.0.1:51804)
0809 18:56:04.573290 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.573326 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:04.573721 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:04.573869 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:04.573871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:04.574190 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:04.574465 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.574471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.575372 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.575373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.576318 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.576321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.576677 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.576678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.576770 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.577197 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:04.577207 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:04.578604 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:04.579915 (+ 1311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.579922 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.579924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.580182 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.580353 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.580353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.580354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.580451 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.580695 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:04.580698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:04.580796 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 18:56:04.580884 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 18:56:04.581022 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":84,"threads_started":1}
I20260809 18:56:04.581710 28188 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:04.581820 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.573039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.23.218.252:43253)
0809 18:56:04.573228 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.573236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.573269 (+ 33us) server_negotiation.cc:407] Connection header received
0809 18:56:04.573700 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.573707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.573730 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.573826 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.574540 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.575646 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.576560 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.576916 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.576979 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.579153 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.579160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.579160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.579166 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.580536 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.580877 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.580880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.580880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.580940 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.581141 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.581142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.581143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.581307 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.581438 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.581607 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.581682 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:56:04.588197 28195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.581941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:36965 (local address 127.0.0.1:53438)
0809 18:56:04.581970 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.581979 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:04.582012 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:04.582127 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:04.582127 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:04.582261 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:04.582300 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.582301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.583061 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.583062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.583746 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.583748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.584152 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.584153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.584198 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.584514 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:04.584518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:04.585818 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:04.587345 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.587346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.587346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.587507 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.587686 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.587687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.587687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.587758 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.588003 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:04.588003 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:04.588035 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 18:56:04.588078 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 18:56:04.588143 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 18:56:04.588255 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.581994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.23.218.253:36965)
0809 18:56:04.582036 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.582038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.582046 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:56:04.582065 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.582066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.582074 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.582124 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.582385 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.582999 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.583840 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.584081 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.584150 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.585915 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.585916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.585916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.585923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.587260 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.587571 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.587572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.587572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.587628 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.587816 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.587817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.587817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.587942 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.588004 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.588104 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.588173 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 18:56:04.588507 28188 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:04.589393 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.582672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51820 (local address 127.23.218.254:33255)
0809 18:56:04.582706 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.582707 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.582750 (+ 43us) server_negotiation.cc:407] Connection header received
0809 18:56:04.582806 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.582807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.582815 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.582862 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.583265 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.583912 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.584622 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.584894 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.584963 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.586986 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.586988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.586988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.586995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.588367 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.588710 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.588711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.588712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.588767 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.588953 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.588953 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.588954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.589054 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.589213 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.589247 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.589311 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
I20260809 18:56:04.594373 28195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.588698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56352)
0809 18:56:04.588731 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.588741 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:04.588784 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:04.588871 (+ 87us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:04.588872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:04.588983 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:04.589006 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.589007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.589592 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.589593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.590285 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.590288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.590690 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.590691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.590729 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.591044 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:04.591050 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:04.592202 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:04.593535 (+ 1333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.593535 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.593536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.593679 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.593829 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.593830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.593830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.593882 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.594076 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:04.594077 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:04.594113 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 18:56:04.594162 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 18:56:04.594237 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 18:56:04.594522 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.588756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56352 (local address 127.23.218.252:43253)
0809 18:56:04.588783 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.588784 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.588791 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:56:04.588822 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.588823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.588830 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.588869 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.589062 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.589534 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.590385 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.590615 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.590676 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.592287 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.592289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.592289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.592295 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.593466 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.593732 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.593733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.593734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.593780 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.593928 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.593928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.593929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.594023 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.594081 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.594203 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.594300 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":15}
I20260809 18:56:04.595877 28188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253: Bootstrap starting.
I20260809 18:56:04.596541 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.589873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.23.218.253:36965)
0809 18:56:04.589904 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.589905 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.589913 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:56:04.589955 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.589956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.589963 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.590002 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.590525 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.591019 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.591786 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.592062 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.592112 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.594053 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.594054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.594055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.594061 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.595523 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.595818 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.595819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.595820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.595879 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.596048 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.596049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.596049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.596173 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.596344 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.596388 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.596464 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 18:56:04.596819 28188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253: Neither blocks nor log segments found. Creating new log.
I20260809 18:56:04.597100 28188 log.cc:824] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253: Log is configured to *not* fsync() on all Append() calls
I20260809 18:56:04.597792 28188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253: No bootstrap required, opened a new log
I20260809 18:56:04.597832 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:56:04.599222 28188 raft_consensus.cc:348] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } }
I20260809 18:56:04.599335 28188 raft_consensus.cc:729] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c26713c9cef240e7a1210865a7090253, State: Initialized, Role: FOLLOWER
I20260809 18:56:04.599391 28188 consensus_queue.cc:260] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } }
I20260809 18:56:04.598845 27894 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "78ae6826c04e44f6b86e375edc481e79"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51820
I20260809 18:56:04.599637 28201 sys_catalog.cc:455] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.599689 28188 sys_catalog.cc:564] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:56:04.599736 28201 sys_catalog.cc:458] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 18:56:04.601193 28221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:56:04.601246 28221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:56:04.601421 28074 master_runner.cc:186] Error getting master registration for 127.23.218.254:33255: OK, instance_id { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" instance_seqno: 1786301764284879 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 18:56:04.603304 28074 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 18:56:04.623219 27934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } attempt: 2
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
I20260809 18:56:04.629320 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.623459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56368)
0809 18:56:04.623582 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.623591 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:04.623637 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:04.623748 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:04.623749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:04.623883 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:04.623915 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.623916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.624459 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.624460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.624951 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.624952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.625238 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.625239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.625290 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.625629 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:04.625634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:04.626858 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:04.628455 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.628456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.628456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.628605 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.628762 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.628763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.628764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.628824 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.629044 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:04.629045 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:04.629086 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 18:56:04.629144 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 18:56:04.629230 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 18:56:04.629320 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.623545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.23.218.252:43253)
0809 18:56:04.623578 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.623579 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.623622 (+ 43us) server_negotiation.cc:407] Connection header received
0809 18:56:04.623680 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.623681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.623689 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.623734 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.623974 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.624396 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.625015 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.625184 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.625236 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.626929 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.626930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.626930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.626937 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.628370 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.628660 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.628662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.628662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.628712 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.628871 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.628872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.628872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.628988 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.629041 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.629171 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.629233 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 18:56:04.630847 27934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79: Bootstrap starting.
I20260809 18:56:04.631417 27934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79: Neither blocks nor log segments found. Creating new log.
I20260809 18:56:04.631654 27934 log.cc:824] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79: Log is configured to *not* fsync() on all Append() calls
I20260809 18:56:04.632277 27934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79: No bootstrap required, opened a new log
I20260809 18:56:04.633206 27934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } }
I20260809 18:56:04.633316 27934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78ae6826c04e44f6b86e375edc481e79, State: Initialized, Role: FOLLOWER
I20260809 18:56:04.633387 27934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } }
I20260809 18:56:04.633616 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.633662 27934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:56:04.633703 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:04.633809 27894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 18:56:04.633893 27894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:56:04.634655 27894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } }
I20260809 18:56:04.635231 28142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78ae6826c04e44f6b86e375edc481e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c26713c9cef240e7a1210865a7090253"
I20260809 18:56:04.635362 28142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [term 0 FOLLOWER]: Advancing to term 1
W20260809 18:56:04.635483 28243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:56:04.635577 28243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:56:04.636154 28142 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78ae6826c04e44f6b86e375edc481e79 in term 1.
I20260809 18:56:04.636682 27862 leader_election.cc:302] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78ae6826c04e44f6b86e375edc481e79, c26713c9cef240e7a1210865a7090253; no voters:
I20260809 18:56:04.636808 28225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:56:04.636940 28225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [term 1 LEADER]: Becoming Leader. State: Replica: 78ae6826c04e44f6b86e375edc481e79, State: Running, Role: LEADER
I20260809 18:56:04.637013 28225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } }
I20260809 18:56:04.637396 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78ae6826c04e44f6b86e375edc481e79. Latest consensus state: current_term: 1 leader_uuid: "78ae6826c04e44f6b86e375edc481e79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.637477 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:04.637601 27894 leader_election.cc:288] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [CANDIDATE]: Term 1 election: Requested vote from peers 2a8b1aa291444e8dba62cbe78cf6467c (127.23.218.253:36965), c26713c9cef240e7a1210865a7090253 (127.23.218.252:43253)
I20260809 18:56:04.636255 27825 master_runner.cc:186] Error getting master registration for 127.23.218.253:36965: OK, instance_id { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" instance_seqno: 1786301764429584 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 18:56:04.638556 28249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:56:04.638594 27825 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 18:56:04.638725 28249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:56:04.640520 28142 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [term 1 FOLLOWER]: Refusing update from remote peer 78ae6826c04e44f6b86e375edc481e79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:56:04.640964 28225 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:56:04.642333 28201 sys_catalog.cc:455] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78ae6826c04e44f6b86e375edc481e79. Latest consensus state: current_term: 1 leader_uuid: "78ae6826c04e44f6b86e375edc481e79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.642443 28201 sys_catalog.cc:458] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:04.645920 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78ae6826c04e44f6b86e375edc481e79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.645992 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:04.646402 28201 sys_catalog.cc:455] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78ae6826c04e44f6b86e375edc481e79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.646478 28201 sys_catalog.cc:458] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:04.646680 28259 mvcc.cc:205] Tried to move back new op lower bound from 7316692027965112320 to 7316692027953844224. Current Snapshot: MvccSnapshot[applied={T|T < 7316692027965112320}]
I20260809 18:56:04.648394 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.639551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51822 (local address 127.23.218.254:33255)
0809 18:56:04.639585 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.639586 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.640391 (+ 805us) server_negotiation.cc:407] Connection header received
0809 18:56:04.640488 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.640489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.640498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.640524 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.642000 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.642672 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.643681 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.643942 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.644008 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.645344 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.645345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.645346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.645352 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.647223 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.647582 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.647583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.647583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.647630 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.647822 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.647823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.647824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.647941 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.648030 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.648227 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.648310 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:56:04.646845 28255 mvcc.cc:205] Tried to move back new op lower bound from 7316692027965112320 to 7316692027953844224. Current Snapshot: MvccSnapshot[applied={T|T < 7316692027965112320}]
I20260809 18:56:04.649423 28249 catalog_manager.cc:1347] Generated new cluster ID: ce0501737671420cae45b31e13990281
I20260809 18:56:04.649477 28249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:56:04.655465 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.639879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.23.218.253:36965)
0809 18:56:04.639919 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.639920 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.642478 (+ 2558us) server_negotiation.cc:407] Connection header received
0809 18:56:04.642595 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.642596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.642608 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.642662 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.646987 (+ 4325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.647642 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.649572 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.649900 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.649942 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.652109 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.652110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.652111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.652118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.653650 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.654330 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.654332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.654332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.654424 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.654723 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.654724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.654724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.654868 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.654961 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.655297 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.655374 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:56:04.657624 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.640039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.23.218.252:43253)
0809 18:56:04.640072 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.640073 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.646501 (+ 6428us) server_negotiation.cc:407] Connection header received
0809 18:56:04.646594 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.646594 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.646604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.646633 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.647179 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.647888 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.651257 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.651534 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.651595 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.654586 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.654588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.654588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.654594 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.656119 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.656877 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.656878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.656878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.656934 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.657144 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.657146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.657146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.657244 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.657325 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.657477 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.657544 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 18:56:04.660161 28056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } attempt: 3
Aug 09 18:56:04 dist-test-slave-6zbq krb5kdc[27806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301764, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
I20260809 18:56:04.667083 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.660394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:43253 (local address 127.0.0.1:56386)
0809 18:56:04.660432 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:04.660441 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:04.660475 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:04.660695 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:04.660696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:04.660866 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:04.660904 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.660906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.661476 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.661477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.662240 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:04.662242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.662694 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.662695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:04.662750 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.663092 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:04.663096 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:04.664482 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:04.666078 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.666079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.666079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.666276 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.666460 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:04.666461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:04.666461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:04.666531 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:04.666800 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:04.666801 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:04.666857 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 18:56:04.666911 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 18:56:04.666995 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 18:56:04.667094 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:04.660536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56386 (local address 127.23.218.252:43253)
0809 18:56:04.660583 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.660584 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.660591 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:56:04.660607 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.660608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.660616 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.660660 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.660969 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.661407 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.662321 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:04.662631 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:04.662684 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:04.664555 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:04.664556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:04.664557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:04.664563 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:04.665999 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.666332 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.666333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.666333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.666406 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:04.666576 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:04.666577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:04.666577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:04.666706 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:04.666758 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:04.666960 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 18:56:04.667019 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 18:56:04.668689 28056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c: Bootstrap starting.
I20260809 18:56:04.669401 28056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c: Neither blocks nor log segments found. Creating new log.
I20260809 18:56:04.669682 28056 log.cc:824] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c: Log is configured to *not* fsync() on all Append() calls
I20260809 18:56:04.670450 28056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c: No bootstrap required, opened a new log
I20260809 18:56:04.672806 28056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } }
I20260809 18:56:04.672935 28056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8b1aa291444e8dba62cbe78cf6467c, State: Initialized, Role: FOLLOWER
I20260809 18:56:04.673014 28056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } }
I20260809 18:56:04.673313 28056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:56:04.673383 28012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78ae6826c04e44f6b86e375edc481e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2a8b1aa291444e8dba62cbe78cf6467c"
I20260809 18:56:04.673491 28012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:56:04.674257 28012 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78ae6826c04e44f6b86e375edc481e79 in term 1.
I20260809 18:56:04.674481 28011 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [term 1 FOLLOWER]: Refusing update from remote peer 78ae6826c04e44f6b86e375edc481e79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:56:04.673297 28270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.674866 28246 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:56:04.674930 28270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:04.676494 28270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78ae6826c04e44f6b86e375edc481e79. Latest consensus state: current_term: 1 leader_uuid: "78ae6826c04e44f6b86e375edc481e79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.676615 28270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:04.676692 28270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78ae6826c04e44f6b86e375edc481e79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.676788 28270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 18:56:04.677603 28286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:56:04.677659 28286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 18:56:04.677712 27982 tablet.cc:2369] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 18:56:04.677970 28249 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:56:04.678557 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78ae6826c04e44f6b86e375edc481e79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78ae6826c04e44f6b86e375edc481e79" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 33255 } } peers { permanent_uuid: "2a8b1aa291444e8dba62cbe78cf6467c" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 36965 } } peers { permanent_uuid: "c26713c9cef240e7a1210865a7090253" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 43253 } } }
I20260809 18:56:04.678637 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79 [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:04.678643 28249 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:56:04.679200 28287 mvcc.cc:205] Tried to move back new op lower bound from 7316692027965112320 to 7316692027953844224. Current Snapshot: MvccSnapshot[applied={T|T < 7316692027965112320}]
I20260809 18:56:04.688795 28249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78ae6826c04e44f6b86e375edc481e79: Generated new TSK 0
I20260809 18:56:04.688948 28249 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 18:56:04.719169 28194 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.718097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56390 (local address 127.23.218.252:43253)
0809 18:56:04.718136 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.718138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.718570 (+ 432us) server_negotiation.cc:407] Connection header received
0809 18:56:04.718594 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.718595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.718604 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.718648 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.719105 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56390: BlockingRecv error: recv got EOF from 127.0.0.1:56390 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 18:56:04.719305 27940 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.717697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.23.218.254:33255)
0809 18:56:04.717742 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.717744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.718385 (+ 641us) server_negotiation.cc:407] Connection header received
0809 18:56:04.718463 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.718464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.718475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.718537 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.719225 (+ 688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51838: BlockingRecv error: recv got EOF from 127.0.0.1:51838 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 18:56:04.722683 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.717895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53460 (local address 127.23.218.253:36965)
0809 18:56:04.717934 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.717935 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.718635 (+ 700us) server_negotiation.cc:407] Connection header received
0809 18:56:04.718726 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.718727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.718737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.718764 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.722625 (+ 3861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53460: BlockingRecv error: recv got EOF from 127.0.0.1:53460 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 18:56:04.724888 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.724144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.23.218.253:36965)
0809 18:56:04.724178 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.724179 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.724420 (+ 241us) server_negotiation.cc:407] Connection header received
0809 18:56:04.724517 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.724519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.724535 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.724563 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.724845 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53464: BlockingRecv error: recv got EOF from 127.0.0.1:53464 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 18:56:04.727805 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.727109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.23.218.253:36965)
0809 18:56:04.727139 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.727141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.727365 (+ 224us) server_negotiation.cc:407] Connection header received
0809 18:56:04.727451 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.727452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.727461 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.727487 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.727765 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53474: BlockingRecv error: recv got EOF from 127.0.0.1:53474 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 18:56:04.731585 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.730905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53480 (local address 127.23.218.253:36965)
0809 18:56:04.730935 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.730936 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.731137 (+ 201us) server_negotiation.cc:407] Connection header received
0809 18:56:04.731221 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.731222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.731230 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.731269 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.731545 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53480: BlockingRecv error: recv got EOF from 127.0.0.1:53480 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 18:56:04.737293 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.736620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53494 (local address 127.23.218.253:36965)
0809 18:56:04.736650 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.736652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.736851 (+ 199us) server_negotiation.cc:407] Connection header received
0809 18:56:04.736935 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.736936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.736944 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.736970 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.737241 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53494: BlockingRecv error: recv got EOF from 127.0.0.1:53494 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 18:56:04.747069 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.746371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53504 (local address 127.23.218.253:36965)
0809 18:56:04.746402 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.746404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.746615 (+ 211us) server_negotiation.cc:407] Connection header received
0809 18:56:04.746701 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.746702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.746711 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.746736 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.747025 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53504: BlockingRecv error: recv got EOF from 127.0.0.1:53504 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 18:56:04.765029 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.764225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53518 (local address 127.23.218.253:36965)
0809 18:56:04.764271 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.764273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.764515 (+ 242us) server_negotiation.cc:407] Connection header received
0809 18:56:04.764612 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.764613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.764623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.764651 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.764986 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53518: BlockingRecv error: recv got EOF from 127.0.0.1:53518 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 18:56:04.799288 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.798468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.23.218.253:36965)
0809 18:56:04.798513 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.798515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.798687 (+ 172us) server_negotiation.cc:407] Connection header received
0809 18:56:04.798750 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.798751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.798765 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.798845 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.799228 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53534: BlockingRecv error: recv got EOF from 127.0.0.1:53534 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 18:56:04.864797 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.864223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53542 (local address 127.23.218.253:36965)
0809 18:56:04.864282 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.864284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.864370 (+ 86us) server_negotiation.cc:407] Connection header received
0809 18:56:04.864425 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.864426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.864435 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.864492 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.864751 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53542: BlockingRecv error: recv got EOF from 127.0.0.1:53542 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 18:56:04.994520 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:04.993877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.23.218.253:36965)
0809 18:56:04.993937 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:04.993939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:04.994027 (+ 88us) server_negotiation.cc:407] Connection header received
0809 18:56:04.994077 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:04.994079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:04.994089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:04.994145 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:04.994458 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53548: BlockingRecv error: recv got EOF from 127.0.0.1:53548 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 18:56:05.251940 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:05.251446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53558 (local address 127.23.218.253:36965)
0809 18:56:05.251495 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:05.251497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:05.251509 (+ 12us) server_negotiation.cc:407] Connection header received
0809 18:56:05.251570 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:05.251571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:05.251580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:05.251642 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:05.251889 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53558: BlockingRecv error: recv got EOF from 127.0.0.1:53558 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 18:56:05.601643 28221 catalog_manager.cc:1259] Loaded cluster ID: ce0501737671420cae45b31e13990281
I20260809 18:56:05.601704 28221 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253: loading cluster ID for follower catalog manager: success
I20260809 18:56:05.602525 28221 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253: acquiring CA information for follower catalog manager: success
I20260809 18:56:05.602826 28221 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c26713c9cef240e7a1210865a7090253: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 18:56:05.678107 28286 catalog_manager.cc:1259] Loaded cluster ID: ce0501737671420cae45b31e13990281
I20260809 18:56:05.678167 28286 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c: loading cluster ID for follower catalog manager: success
I20260809 18:56:05.679061 28286 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c: acquiring CA information for follower catalog manager: success
I20260809 18:56:05.679371 28286 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a8b1aa291444e8dba62cbe78cf6467c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 18:56:05.767266 28392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:05.765032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37628 (local address 127.23.218.253:36965)
0809 18:56:05.765140 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:05.765142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:05.765152 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:56:05.765210 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:05.765211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:05.765262 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:05.765338 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:05.765459 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.765943 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.766669 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.766847 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.766886 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:05.766992 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:05.767099 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:05.767150 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:05.767165 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 18:56:05.767181 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260809 18:56:05.771003 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:05.768921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49482 (local address 127.23.218.252:43253)
0809 18:56:05.769020 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:05.769022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:05.769029 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:56:05.769063 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:05.769064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:05.769097 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:05.769163 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:05.769300 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.769760 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.770445 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.770634 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.770678 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:05.770780 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:05.770865 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:05.770914 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:05.770930 (+ 16us) server_negotiation.cc:299] Negotiation successful
0809 18:56:05.770946 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":40,"threads_started":1}
I20260809 18:56:05.775254 28392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:05.772548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37638 (local address 127.23.218.253:36965)
0809 18:56:05.772590 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:05.772592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:05.772655 (+ 63us) server_negotiation.cc:407] Connection header received
0809 18:56:05.772753 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:05.772754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:05.772765 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:05.772814 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:05.772925 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.773506 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.774612 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.774811 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.774861 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:05.774997 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:05.775026 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:05.775053 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:05.775169 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 18:56:05.775190 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:56:05.775379 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:05.772433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49498 (local address 127.23.218.252:43253)
0809 18:56:05.772514 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:05.772516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:05.772539 (+ 23us) server_negotiation.cc:407] Connection header received
0809 18:56:05.772604 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:05.772605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:05.772617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:05.772788 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:05.772819 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.773587 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.774563 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.774856 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.774947 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:05.775071 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:05.775102 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:05.775169 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:05.775220 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 18:56:05.775238 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 18:56:05.775616 28420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:05.772410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40376 (local address 127.23.218.254:33255)
0809 18:56:05.772616 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:05.772619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:05.772643 (+ 24us) server_negotiation.cc:407] Connection header received
0809 18:56:05.772717 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:05.772718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:05.772765 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:05.772835 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:05.773017 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.773759 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.774660 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:05.774963 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:05.775016 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:05.775185 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:05.775280 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:05.775334 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:05.775513 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 18:56:05.775539 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":62,"threads_started":1}
I20260809 18:56:05.777105 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27825
I20260809 18:56:05.788947 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 27951
I20260809 18:56:05.801281 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 28074
2026-08-09T18:56:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2871 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 18:56:05 dist-test-slave-6zbq krb5kdc[28426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 18:56:05 dist-test-slave-6zbq krb5kdc[28426](info): set up 2 sockets
Aug 09 18:56:05 dist-test-slave-6zbq krb5kdc[28426](info): commencing operation
krb5kdc: starting...
W20260809 18:56:06.934988 24427 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/test-user.keytab.
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T18:56:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T18:56:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.218.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
Entry for principal kudu/127.23.218.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab.
I20260809 18:56:07.039068 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36105
--webserver_interface=127.23.218.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:36105,127.23.218.253:34085,127.23.218.252:38907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 18:56:07.116873 28445 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 18:56:07.117064 28445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:56:07.117089 28445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:56:07.117112 28445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:56:07.118228 28445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:56:07.118266 28445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:56:07.118285 28445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:56:07.118300 28445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:56:07.118316 28445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:56:07.119792 28445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:36105,127.23.218.253:34085,127.23.218.252:38907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.254.keytab
--principal=kudu/127.23.218.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.254:36105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.218.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:56:07.120019 28445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:56:07.120203 28445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:56:07.124627 28452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:07.124730 28445 server_base.cc:1034] running on GCE node
W20260809 18:56:07.124645 28454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:07.124662 28457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:07.125085 28445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:56:07.125341 28445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:56:07.126470 28445 hybrid_clock.cc:648] HybridClock initialized: now 1786301767126456 us; error 29 us; skew 500 ppm
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:56:07.133390 28445 init.cc:377] Logged in from keytab as kudu/127.23.218.254@KRBTEST.COM (short username kudu)
I20260809 18:56:07.133767 28445 webserver.cc:460] Webserver started at http://127.23.218.254:34691/ using document root <none> and password file <none>
I20260809 18:56:07.133955 28445 fs_manager.cc:359] Metadata directory not provided
I20260809 18:56:07.134001 28445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:56:07.134119 28445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:56:07.135123 28445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/data/instance:
uuid: "f22573f41dfc448e9d8cc833fe596b5c"
format_stamp: "Formatted at 2026-08-09 18:56:07 on dist-test-slave-6zbq"
server_key: "fda753929d7b4dede3f0a1800621ac72"
server_key_iv: "726b754bcdbef64ba7cd1823acc32cd3"
server_key_version: "encryptionkey@0"
I20260809 18:56:07.135442 28445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/wal/instance:
uuid: "f22573f41dfc448e9d8cc833fe596b5c"
format_stamp: "Formatted at 2026-08-09 18:56:07 on dist-test-slave-6zbq"
server_key: "fda753929d7b4dede3f0a1800621ac72"
server_key_iv: "726b754bcdbef64ba7cd1823acc32cd3"
server_key_version: "encryptionkey@0"
I20260809 18:56:07.136596 28445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 18:56:07.137205 28466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:07.137352 28445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:56:07.137434 28445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/wal
uuid: "f22573f41dfc448e9d8cc833fe596b5c"
format_stamp: "Formatted at 2026-08-09 18:56:07 on dist-test-slave-6zbq"
server_key: "fda753929d7b4dede3f0a1800621ac72"
server_key_iv: "726b754bcdbef64ba7cd1823acc32cd3"
server_key_version: "encryptionkey@0"
I20260809 18:56:07.137497 28445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:56:07.160040 28445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:56:07.161924 28445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:56:07.162048 28445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:56:07.165669 28445 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.254:36105
I20260809 18:56:07.165710 28560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.254:36105 every 8 connection(s)
I20260809 18:56:07.165985 28445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/data/info.pb
I20260809 18:56:07.166769 28561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:56:07.168088 28561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:07.173010 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 28445
I20260809 18:56:07.173105 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-0/wal/instance
I20260809 18:56:07.173280 24427 external_mini_cluster.cc:1428] Setting key d78d79b8b75167c7c9da8baa2c0b8658
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:56:07.176703 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.168627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36105 (local address 127.0.0.1:47130)
0809 18:56:07.168775 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.168806 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:07.169208 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:07.169425 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:07.169427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:07.169636 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:07.169789 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.169793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.170742 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.170742 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.171403 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.171406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.172040 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.172041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.172121 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.172493 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:07.172500 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:07.173830 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:07.175420 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.175427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.175430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.175674 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.175873 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.175874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.175874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.175974 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.176342 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:07.176345 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:07.176395 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 18:56:07.176457 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 18:56:07.176566 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260809 18:56:07.176738 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.168778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.23.218.254:36105)
0809 18:56:07.168896 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.168901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.168928 (+ 27us) server_negotiation.cc:407] Connection header received
0809 18:56:07.169322 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.169323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.169340 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.169410 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.169854 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.170672 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.171475 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.171948 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.172063 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.173908 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.173917 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.173917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.173924 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.175344 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.175742 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.175743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.175744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.175820 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.176034 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.176035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.176035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.176275 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.176351 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.176506 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.176642 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":88,"thread_start_us":43,"threads_started":1}
I20260809 18:56:07.177384 28561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:07.177680 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.177571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:34085 (local address 127.0.0.1:35736)
0809 18:56:07.177615 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.177658 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 18:56:07.177798 28474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
I20260809 18:56:07.178165 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.178068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:34085 (local address 127.0.0.1:35752)
0809 18:56:07.178114 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.178143 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 18:56:07.178269 28561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.253:34085: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.23.218.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab.
Entry for principal kudu/127.23.218.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab.
I20260809 18:56:07.194300 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.253:34085
--webserver_interface=127.23.218.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab
--principal=kudu/127.23.218.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:36105,127.23.218.253:34085,127.23.218.252:38907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:56:07.227495 28561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } attempt: 1
I20260809 18:56:07.227867 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.227769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:34085 (local address 127.0.0.1:35762)
0809 18:56:07.227816 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.227846 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 18:56:07.228157 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.228094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:34085 (local address 127.0.0.1:35772)
0809 18:56:07.228124 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.228144 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 18:56:07.228237 28561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.253:34085: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
W20260809 18:56:07.272962 28573 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 18:56:07.273145 28573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:56:07.273162 28573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:56:07.273186 28573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:56:07.274250 28573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:56:07.274281 28573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:56:07.274292 28573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:56:07.274303 28573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:56:07.274317 28573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:56:07.275758 28573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:36105,127.23.218.253:34085,127.23.218.252:38907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.253.keytab
--principal=kudu/127.23.218.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.253:34085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.218.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:56:07.275933 28573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:56:07.276095 28573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 18:56:07.280320 28579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:07.280344 28586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:07.280512 28573 server_base.cc:1034] running on GCE node
W20260809 18:56:07.280536 28580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:07.280772 28573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:56:07.281030 28573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:56:07.282207 28573 hybrid_clock.cc:648] HybridClock initialized: now 1786301767282186 us; error 32 us; skew 500 ppm
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:56:07.284390 28573 init.cc:377] Logged in from keytab as kudu/127.23.218.253@KRBTEST.COM (short username kudu)
I20260809 18:56:07.284804 28573 webserver.cc:460] Webserver started at http://127.23.218.253:42535/ using document root <none> and password file <none>
I20260809 18:56:07.285006 28573 fs_manager.cc:359] Metadata directory not provided
I20260809 18:56:07.285051 28573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:56:07.285172 28573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:56:07.286092 28573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/data/instance:
uuid: "ddb7a0fe16d942479a476f0add03a597"
format_stamp: "Formatted at 2026-08-09 18:56:07 on dist-test-slave-6zbq"
server_key: "0f7e1f554fb9dca1e33639c31b94f70c"
server_key_iv: "cb43d5723eb1e12f21de8aa6b8a376e8"
server_key_version: "encryptionkey@0"
I20260809 18:56:07.286445 28573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/wal/instance:
uuid: "ddb7a0fe16d942479a476f0add03a597"
format_stamp: "Formatted at 2026-08-09 18:56:07 on dist-test-slave-6zbq"
server_key: "0f7e1f554fb9dca1e33639c31b94f70c"
server_key_iv: "cb43d5723eb1e12f21de8aa6b8a376e8"
server_key_version: "encryptionkey@0"
I20260809 18:56:07.287576 28573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 18:56:07.288182 28600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:07.288344 28573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:07.288414 28573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/wal
uuid: "ddb7a0fe16d942479a476f0add03a597"
format_stamp: "Formatted at 2026-08-09 18:56:07 on dist-test-slave-6zbq"
server_key: "0f7e1f554fb9dca1e33639c31b94f70c"
server_key_iv: "cb43d5723eb1e12f21de8aa6b8a376e8"
server_key_version: "encryptionkey@0"
I20260809 18:56:07.288465 28573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:56:07.296383 28561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } attempt: 2
I20260809 18:56:07.296689 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.296583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:34085 (local address 127.0.0.1:35788)
0809 18:56:07.296632 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.296663 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 18:56:07.296950 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.296885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:34085 (local address 127.0.0.1:35796)
0809 18:56:07.296914 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.296935 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 18:56:07.297019 28561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.253:34085: Network error: Client connection negotiation failed: client connection to 127.23.218.253:34085: connect: Connection refused (error 111)
I20260809 18:56:07.306257 28573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:56:07.307739 28573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:56:07.307834 28573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:56:07.311604 28573 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.253:34085
I20260809 18:56:07.311643 28687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.253:34085 every 8 connection(s)
I20260809 18:56:07.311921 28573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/data/info.pb
I20260809 18:56:07.312640 28688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:56:07.313936 28688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:07.318677 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 28573
I20260809 18:56:07.318780 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-1/wal/instance
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:56:07.318974 24427 external_mini_cluster.cc:1428] Setting key 2554357f6593f68bc91c13e931bedd26
I20260809 18:56:07.321753 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.314493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47140 (local address 127.23.218.254:36105)
0809 18:56:07.314542 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.314544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.314727 (+ 183us) server_negotiation.cc:407] Connection header received
0809 18:56:07.315167 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.315167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.315182 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.315227 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.315769 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.316234 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.317120 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.317319 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.317372 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.319308 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.319310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.319310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.319316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.320466 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.320844 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.320845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.320846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.320906 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.321112 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.321113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.321114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.321246 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.321318 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.321599 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.321681 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:56:07.321805 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.314457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36105 (local address 127.0.0.1:47140)
0809 18:56:07.314623 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.314654 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:07.315079 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:07.315291 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:07.315294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:07.315514 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:07.315684 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.315688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.316311 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.316312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.317038 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.317040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.317386 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.317387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.317505 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.317894 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:07.317901 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:07.319224 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:07.320551 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.320558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.320561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.320779 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.320964 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.320965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.320965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.321054 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.321327 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:07.321334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:07.321474 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 18:56:07.321556 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 18:56:07.321662 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260809 18:56:07.322314 28688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.218.252@KRBTEST.COM; defaulting to no policy
I20260809 18:56:07.328832 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.322534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:34085 (local address 127.0.0.1:35798)
0809 18:56:07.322577 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.322586 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:07.322628 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:07.322892 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:07.322893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:07.322991 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:07.323045 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.323047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.324234 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.324234 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.324690 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.324691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.325292 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.325293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.325343 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.325653 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:07.325657 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:07.326687 (+ 1030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:07.328078 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.328078 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.328079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.328236 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.328372 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.328373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.328373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.328414 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.328592 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:07.328592 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:07.328624 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 18:56:07.328687 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 18:56:07.328777 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 18:56:07.328935 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.322580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.23.218.253:34085)
0809 18:56:07.322727 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.322731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.322760 (+ 29us) server_negotiation.cc:407] Connection header received
0809 18:56:07.322785 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.322788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.322802 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.322880 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.323123 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.324157 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.324761 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.325206 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.325297 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.326764 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.326772 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.326773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.326779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.328011 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.328286 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.328288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.328288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.328332 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.328447 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.328447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.328447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.328549 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.328612 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.328712 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.328836 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.218.252@KRBTEST.COM" created.
I20260809 18:56:07.329456 28688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:07.329741 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.329636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57464)
0809 18:56:07.329683 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.329723 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 18:56:07.329871 28607 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
I20260809 18:56:07.330243 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.330158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57472)
0809 18:56:07.330199 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.330222 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 18:56:07.330369 28688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38907: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.218.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab.
Entry for principal kudu/127.23.218.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab.
I20260809 18:56:07.341747 24427 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
/tmp/dist-test-taskUXKsiJ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.252:38907
--webserver_interface=127.23.218.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab
--principal=kudu/127.23.218.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.218.212:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.218.254:36105,127.23.218.253:34085,127.23.218.252:38907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 18:56:07.379550 28688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } attempt: 1
I20260809 18:56:07.379887 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.379794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57482)
0809 18:56:07.379839 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.379865 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 18:56:07.380139 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.380071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57494)
0809 18:56:07.380103 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.380124 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 18:56:07.380215 28688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38907: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
I20260809 18:56:07.388144 28561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } attempt: 3
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
I20260809 18:56:07.395089 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.388366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:34085 (local address 127.0.0.1:35806)
0809 18:56:07.388408 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.388420 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:07.388465 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:07.388622 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:07.388622 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:07.388748 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:07.388813 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.388815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.389664 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.389665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.390173 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.390174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.390676 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.390677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.390741 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.391131 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:07.391135 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:07.392271 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:07.393797 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.393798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.393798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.393952 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.394190 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.394191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.394191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.394318 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.394650 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:07.394652 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:07.394733 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 18:56:07.394826 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 18:56:07.394970 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 18:56:07.395512 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.388456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35806 (local address 127.23.218.253:34085)
0809 18:56:07.388508 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.388510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.388518 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:56:07.388541 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.388542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.388552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.388617 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.388877 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.389598 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.390259 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.390608 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.390675 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.392343 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.392345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.392345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.392352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.393722 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.394011 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.394012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.394012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.394068 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.394423 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.394424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.394424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.394558 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.394648 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.395350 (+ 702us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.395430 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 18:56:07.395859 28561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:07.396152 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.396066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57508)
0809 18:56:07.396103 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.396125 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 18:56:07.396395 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.396333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57516)
0809 18:56:07.396360 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.396377 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260809 18:56:07.396469 28561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38907: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
W20260809 18:56:07.420950 28699 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 18:56:07.421130 28699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 18:56:07.421149 28699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 18:56:07.421181 28699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 18:56:07.422288 28699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 18:56:07.422323 28699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 18:56:07.422361 28699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 18:56:07.422387 28699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 18:56:07.422405 28699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 18:56:07.423838 28699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.218.212:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.218.254:36105,127.23.218.253:34085,127.23.218.252:38907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/krb5kdc/kudu_127.23.218.252.keytab
--principal=kudu/127.23.218.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.218.252:38907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.218.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 17:56:31 UTC on ccycloud.quasar-esvprx.root.comops.site
build id 4576688
I20260809 18:56:07.424039 28699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 18:56:07.424228 28699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 18:56:07.427578 28561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } attempt: 1
I20260809 18:56:07.427877 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.427767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57520)
0809 18:56:07.427822 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.427854 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 18:56:07.428234 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.428158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57522)
0809 18:56:07.428187 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.428211 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 18:56:07.428385 28561 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38907: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
W20260809 18:56:07.428424 28709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:07.428472 28713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 18:56:07.428485 28710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 18:56:07.429041 28699 server_base.cc:1034] running on GCE node
I20260809 18:56:07.429250 28699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 18:56:07.429486 28699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 18:56:07.430609 28699 hybrid_clock.cc:648] HybridClock initialized: now 1786301767430597 us; error 23 us; skew 500 ppm
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 18:56:07.432682 28699 init.cc:377] Logged in from keytab as kudu/127.23.218.252@KRBTEST.COM (short username kudu)
I20260809 18:56:07.433084 28699 webserver.cc:460] Webserver started at http://127.23.218.252:44633/ using document root <none> and password file <none>
I20260809 18:56:07.433285 28699 fs_manager.cc:359] Metadata directory not provided
I20260809 18:56:07.433331 28699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 18:56:07.433446 28699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 18:56:07.434415 28699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/data/instance:
uuid: "7d8dea1554dc4519b61eb5a43ff3d558"
format_stamp: "Formatted at 2026-08-09 18:56:07 on dist-test-slave-6zbq"
server_key: "a2e68b7585eb58582c10d994eef425aa"
server_key_iv: "278d4cd9903a30add9634f5368cfd9f9"
server_key_version: "encryptionkey@0"
I20260809 18:56:07.434724 28699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/wal/instance:
uuid: "7d8dea1554dc4519b61eb5a43ff3d558"
format_stamp: "Formatted at 2026-08-09 18:56:07 on dist-test-slave-6zbq"
server_key: "a2e68b7585eb58582c10d994eef425aa"
server_key_iv: "278d4cd9903a30add9634f5368cfd9f9"
server_key_version: "encryptionkey@0"
I20260809 18:56:07.435844 28699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 18:56:07.436470 28724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 18:56:07.436617 28699 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 18:56:07.436703 28699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/data,/tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/wal
uuid: "7d8dea1554dc4519b61eb5a43ff3d558"
format_stamp: "Formatted at 2026-08-09 18:56:07 on dist-test-slave-6zbq"
server_key: "a2e68b7585eb58582c10d994eef425aa"
server_key_iv: "278d4cd9903a30add9634f5368cfd9f9"
server_key_version: "encryptionkey@0"
I20260809 18:56:07.436765 28699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 18:56:07.448338 28688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } attempt: 2
I20260809 18:56:07.448611 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.448525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57530)
0809 18:56:07.448569 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.448591 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 18:56:07.448876 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.448807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57532)
0809 18:56:07.448840 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.448862 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 18:56:07.448968 28688 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.218.252:38907: Network error: Client connection negotiation failed: client connection to 127.23.218.252:38907: connect: Connection refused (error 111)
I20260809 18:56:07.451603 28699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 18:56:07.453061 28699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 18:56:07.453157 28699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 18:56:07.456642 28699 rpc_server.cc:307] RPC server started. Bound to: 127.23.218.252:38907
I20260809 18:56:07.456683 28795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.218.252:38907 every 8 connection(s)
I20260809 18:56:07.456974 28699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/data/info.pb
I20260809 18:56:07.457726 28797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 18:56:07.459066 28797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:56:07.465557 24427 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu as pid 28699
I20260809 18:56:07.465664 24427 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUXKsiJ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786301499543878-24427-0/minicluster-data/master-2/wal/instance
I20260809 18:56:07.465881 24427 external_mini_cluster.cc:1428] Setting key 88cca15fafc17272063af3bec4de0f80
I20260809 18:56:07.466246 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.459594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.23.218.254:36105)
0809 18:56:07.459630 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.459631 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.459812 (+ 181us) server_negotiation.cc:407] Connection header received
0809 18:56:07.460226 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.460226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.460234 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.460273 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.460772 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.461217 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.462044 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.462215 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.462258 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.463933 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.463934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.463935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.463941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.465105 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.465409 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.465410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.465411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.465467 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.465661 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.465661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.465662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.465798 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.465865 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.466106 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.466170 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 18:56:07.466501 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.459565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.254:36105 (local address 127.0.0.1:47148)
0809 18:56:07.459714 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.459749 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:07.460147 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:07.460331 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:07.460335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:07.460536 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:07.460713 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.460717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.461271 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.461272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.461989 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.461991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.462263 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.462264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.462370 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.462704 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:07.462712 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:07.463855 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:07.465163 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.465166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.465168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.465355 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.465526 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.465526 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.465526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.465605 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.465858 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:07.465864 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:07.465985 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 18:56:07.466088 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 18:56:07.466274 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":54,"threads_started":1}
I20260809 18:56:07.466981 28797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
I20260809 18:56:07.473361 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.467167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.253:34085 (local address 127.0.0.1:35812)
0809 18:56:07.467210 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.467221 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:07.467268 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:07.467370 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:07.467371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:07.467501 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:07.467544 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.467546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.468357 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.468358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.469073 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.469076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.469421 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.469422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.469468 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.469804 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:07.469808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:07.471009 (+ 1201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:07.472482 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.472483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.472483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.472690 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.472864 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.472865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.472866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.472927 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.473165 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:07.473166 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:07.473197 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 18:56:07.473240 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 18:56:07.473306 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 18:56:07.473415 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.467219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.23.218.253:34085)
0809 18:56:07.467256 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.467257 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.467266 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:56:07.467318 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.467319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.467326 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.467366 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.467610 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.468291 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.469150 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.469355 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.469413 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.471091 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.471092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.471093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.471100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.472407 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.472755 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.472756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.472756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.472808 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.472984 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.472984 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.472985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.473107 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.473176 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.473267 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.473336 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 18:56:07.473753 28797 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } has no permanent_uuid. Determining permanent_uuid...
I20260809 18:56:07.474797 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.466264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.23.218.252:38907)
0809 18:56:07.466405 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.466409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.466668 (+ 259us) server_negotiation.cc:407] Connection header received
0809 18:56:07.466691 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.466693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.466706 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.466765 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.467432 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.468402 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.469352 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.469784 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.469845 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.472088 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.472096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.472097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.472103 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.473658 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.473984 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.473987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.473988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.474048 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.474226 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.474226 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.474227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.474398 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.474557 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.474622 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.474693 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.252@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.254@KRBTEST.COM
I20260809 18:56:07.480350 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.473969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57562)
0809 18:56:07.474002 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.474012 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:07.474043 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:07.474232 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:07.474233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:07.474334 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:07.474386 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.474388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.475151 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.475151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.475720 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.475722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.476041 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.476042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.476095 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.476425 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:07.476428 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:07.477449 (+ 1021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:07.479427 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.479428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.479428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.479602 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.479799 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.479800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.479800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.479866 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.480105 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:07.480105 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:07.480149 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 18:56:07.480215 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 18:56:07.480296 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 18:56:07.480473 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.473997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.23.218.252:38907)
0809 18:56:07.474119 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.474121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.474127 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:56:07.474145 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.474146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.474153 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.474210 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.474438 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.475087 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.475795 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.475981 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.476026 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.477528 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.477529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.477530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.477535 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.479356 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.479657 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.479658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.479658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.479714 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.479925 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.479926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.479926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.480043 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.480098 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.480246 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.480352 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260809 18:56:07.481804 28797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558: Bootstrap starting.
I20260809 18:56:07.482131 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.475438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47162 (local address 127.23.218.254:36105)
0809 18:56:07.475470 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.475472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.475492 (+ 20us) server_negotiation.cc:407] Connection header received
0809 18:56:07.475545 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.475546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.475553 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.475588 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.476054 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.476605 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.477359 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.477614 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.477660 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.479822 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.479823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.479823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.479829 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.481129 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.481479 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.481480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.481480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.481520 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.481693 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.481693 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.481693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.481775 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.481886 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.482027 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.482077 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 18:56:07.482482 28797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558: Neither blocks nor log segments found. Creating new log.
I20260809 18:56:07.482774 28797 log.cc:824] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558: Log is configured to *not* fsync() on all Append() calls
I20260809 18:56:07.483506 28797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558: No bootstrap required, opened a new log
I20260809 18:56:07.485111 28797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } }
I20260809 18:56:07.485234 28797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d8dea1554dc4519b61eb5a43ff3d558, State: Initialized, Role: FOLLOWER
I20260809 18:56:07.485320 28797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } }
I20260809 18:56:07.485613 28797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:56:07.485716 28812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.485811 28812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.218.253@KRBTEST.COM
W20260809 18:56:07.487366 28832 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:56:07.487425 28832 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:56:07.487635 28699 master_runner.cc:186] Error getting master registration for 127.23.218.254:36105: OK, instance_id { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" instance_seqno: 1786301767164706 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 18:56:07.489574 28699 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 18:56:07.490334 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.482996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35826 (local address 127.23.218.253:34085)
0809 18:56:07.483044 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.483046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.483054 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:56:07.483073 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.483074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.483083 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.483141 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.483659 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.484069 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.484936 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.485150 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.485196 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.487721 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.487722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.487723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.487729 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.489220 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.489719 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.489720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.489721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.489778 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.489974 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.489974 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.489974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.490056 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.490162 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.490229 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.490282 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:56:07.491559 24427 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 18:56:07.492487 28517 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f22573f41dfc448e9d8cc833fe596b5c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47162
I20260809 18:56:07.503520 28561 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } attempt: 2
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.254@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
I20260809 18:56:07.510519 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.503755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57564)
0809 18:56:07.503794 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.503821 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:07.503868 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:07.503999 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:07.504000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:07.504152 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:07.504191 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.504192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.505049 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.505049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.505849 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.505851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.506294 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.506295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.506380 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.506678 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:07.506682 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:07.507853 (+ 1171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:07.509500 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.509501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.509502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.509704 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.509893 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.509894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.509894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.509955 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.510210 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:07.510211 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:07.510252 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 18:56:07.510318 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 18:56:07.510425 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 18:56:07.510519 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.503840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.23.218.252:38907)
0809 18:56:07.503890 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.503891 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.503916 (+ 25us) server_negotiation.cc:407] Connection header received
0809 18:56:07.503935 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.503936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.503944 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.503990 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.504258 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.504978 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.505920 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.506232 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.506289 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.507937 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.507938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.507939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.507945 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.509425 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.509786 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.509787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.509787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.509837 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.510009 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.510009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.510009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.510152 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.510211 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.510360 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.510434 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 18:56:07.512387 28561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c: Bootstrap starting.
I20260809 18:56:07.513062 28561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c: Neither blocks nor log segments found. Creating new log.
I20260809 18:56:07.513355 28561 log.cc:824] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c: Log is configured to *not* fsync() on all Append() calls
I20260809 18:56:07.514114 28561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c: No bootstrap required, opened a new log
I20260809 18:56:07.515434 28561 raft_consensus.cc:348] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } }
I20260809 18:56:07.515563 28561 raft_consensus.cc:729] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22573f41dfc448e9d8cc833fe596b5c, State: Initialized, Role: FOLLOWER
I20260809 18:56:07.515638 28561 consensus_queue.cc:260] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } }
I20260809 18:56:07.515946 28561 sys_catalog.cc:564] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:56:07.515916 28838 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.515971 28517 raft_consensus.cc:482] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 18:56:07.516000 28838 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:07.516167 28517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:56:07.516827 28517 raft_consensus.cc:504] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } }
I20260809 18:56:07.517534 28762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f22573f41dfc448e9d8cc833fe596b5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7d8dea1554dc4519b61eb5a43ff3d558"
I20260809 18:56:07.517644 28762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [term 0 FOLLOWER]: Advancing to term 1
W20260809 18:56:07.517784 28861 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:56:07.517870 28861 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:56:07.518497 28762 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22573f41dfc448e9d8cc833fe596b5c in term 1.
I20260809 18:56:07.518692 28517 leader_election.cc:288] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [CANDIDATE]: Term 1 election: Requested vote from peers ddb7a0fe16d942479a476f0add03a597 (127.23.218.253:34085), 7d8dea1554dc4519b61eb5a43ff3d558 (127.23.218.252:38907)
I20260809 18:56:07.518250 28445 master_runner.cc:186] Error getting master registration for 127.23.218.253:34085: OK, instance_id { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" instance_seqno: 1786301767310583 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 18:56:07.520370 28445 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 18:56:07.520501 28474 leader_election.cc:302] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d8dea1554dc4519b61eb5a43ff3d558, f22573f41dfc448e9d8cc833fe596b5c; no voters:
I20260809 18:56:07.520603 28838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 18:56:07.520747 28838 raft_consensus.cc:686] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [term 1 LEADER]: Becoming Leader. State: Replica: f22573f41dfc448e9d8cc833fe596b5c, State: Running, Role: LEADER
I20260809 18:56:07.520828 28838 consensus_queue.cc:237] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } }
I20260809 18:56:07.521256 28863 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f22573f41dfc448e9d8cc833fe596b5c. Latest consensus state: current_term: 1 leader_uuid: "f22573f41dfc448e9d8cc833fe596b5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.521572 28863 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:07.521934 28878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 18:56:07.522279 28878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 18:56:07.524474 28762 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [term 1 FOLLOWER]: Refusing update from remote peer f22573f41dfc448e9d8cc833fe596b5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:56:07.525053 28838 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:56:07.526361 28812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f22573f41dfc448e9d8cc833fe596b5c. Latest consensus state: current_term: 1 leader_uuid: "f22573f41dfc448e9d8cc833fe596b5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.526445 28812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:07.527459 28838 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f22573f41dfc448e9d8cc833fe596b5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.527532 28838 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:07.527941 28812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f22573f41dfc448e9d8cc833fe596b5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.528009 28812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:07.528187 28888 mvcc.cc:205] Tried to move back new op lower bound from 7316692039777320960 to 7316692039765954560. Current Snapshot: MvccSnapshot[applied={T|T < 7316692039777320960}]
I20260809 18:56:07.528343 28890 mvcc.cc:205] Tried to move back new op lower bound from 7316692039777320960 to 7316692039765954560. Current Snapshot: MvccSnapshot[applied={T|T < 7316692039777320960}]
I20260809 18:56:07.531364 28878 catalog_manager.cc:1347] Generated new cluster ID: ad0c19783401474ab515c13f777f96ce
I20260809 18:56:07.531412 28878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 18:56:07.534142 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.522601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47174 (local address 127.23.218.254:36105)
0809 18:56:07.522640 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.522642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.523486 (+ 844us) server_negotiation.cc:407] Connection header received
0809 18:56:07.523577 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.523577 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.523586 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.523613 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.526674 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.527384 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.528538 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.528857 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.528909 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.530998 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.530999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.531000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.531018 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.532557 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.533120 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.533121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.533121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.533191 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.533497 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.533498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.533499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.533618 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.533711 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.533973 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.534056 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:56:07.539222 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.522769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.23.218.253:34085)
0809 18:56:07.522807 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.522809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.530473 (+ 7664us) server_negotiation.cc:407] Connection header received
0809 18:56:07.530570 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.530571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.530582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.530610 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.531652 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.532562 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.533858 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.534160 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.534192 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.535626 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.535628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.535628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.535636 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.537110 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.538079 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.538080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.538081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.538150 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.538597 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.538612 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.538612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.538744 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.538838 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.539058 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.539133 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 18:56:07.540098 28688 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } attempt: 3
I20260809 18:56:07.543843 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.522932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57568 (local address 127.23.218.252:38907)
0809 18:56:07.522975 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.522976 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.530500 (+ 7524us) server_negotiation.cc:407] Connection header received
0809 18:56:07.530633 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.530634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.530655 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.530681 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.536096 (+ 5415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.536948 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.538099 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.538405 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.538439 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.540201 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.540202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.540203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.540209 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.542413 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.543143 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.543143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.543144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.543192 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.543361 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.543361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.543362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.543455 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.543532 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.543701 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.543766 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 18:56:07 dist-test-slave-6zbq krb5kdc[28426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786301767, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.218.253@KRBTEST.COM for kudu/127.23.218.252@KRBTEST.COM
I20260809 18:56:07.547971 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.540534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.218.252:38907 (local address 127.0.0.1:57582)
0809 18:56:07.540576 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 18:56:07.540584 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 18:56:07.540614 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 18:56:07.540772 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 18:56:07.540773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 18:56:07.540921 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 18:56:07.540958 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.540959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.541836 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.541837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.542612 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 18:56:07.542615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.543040 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.543041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 18:56:07.543094 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.543510 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 18:56:07.543529 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 18:56:07.545150 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 18:56:07.546978 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.546979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.546980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.547202 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.547386 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 18:56:07.547387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 18:56:07.547387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 18:56:07.547457 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 18:56:07.547676 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 18:56:07.547677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 18:56:07.547735 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 18:56:07.547796 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 18:56:07.547877 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 18:56:07.547987 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:07.540561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57582 (local address 127.23.218.252:38907)
0809 18:56:07.540668 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.540669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.540675 (+ 6us) server_negotiation.cc:407] Connection header received
0809 18:56:07.540690 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.540691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.540700 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.540744 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.541061 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.541756 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.542687 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:07.542977 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:07.543030 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:07.545236 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 18:56:07.545237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 18:56:07.545237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 18:56:07.545244 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 18:56:07.546906 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.547282 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.547283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.547284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.547337 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 18:56:07.547505 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 18:56:07.547505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 18:56:07.547506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 18:56:07.547630 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 18:56:07.547686 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:07.547833 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 18:56:07.547904 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 18:56:07.549322 28688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597: Bootstrap starting.
I20260809 18:56:07.549961 28688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597: Neither blocks nor log segments found. Creating new log.
I20260809 18:56:07.550246 28688 log.cc:824] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597: Log is configured to *not* fsync() on all Append() calls
I20260809 18:56:07.551165 28688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597: No bootstrap required, opened a new log
I20260809 18:56:07.553069 28688 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } }
I20260809 18:56:07.553195 28688 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddb7a0fe16d942479a476f0add03a597, State: Initialized, Role: FOLLOWER
I20260809 18:56:07.553301 28688 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } }
I20260809 18:56:07.553591 28688 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 18:56:07.553649 28643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f22573f41dfc448e9d8cc833fe596b5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ddb7a0fe16d942479a476f0add03a597"
I20260809 18:56:07.553763 28643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [term 0 FOLLOWER]: Advancing to term 1
I20260809 18:56:07.554468 28643 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f22573f41dfc448e9d8cc833fe596b5c in term 1.
I20260809 18:56:07.554631 28641 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [term 1 FOLLOWER]: Refusing update from remote peer f22573f41dfc448e9d8cc833fe596b5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 18:56:07.553596 28898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.554976 28863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 18:56:07.555104 28898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 18:56:07.556677 28914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 18:56:07.556725 28914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 18:56:07.557085 28915 mvcc.cc:205] Tried to move back new op lower bound from 7316692039777320960 to 7316692039765954560. Current Snapshot: MvccSnapshot[applied={T|T < 7316692039777320960}]
I20260809 18:56:07.557538 28920 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f22573f41dfc448e9d8cc833fe596b5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.557615 28920 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:07.557880 28898 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f22573f41dfc448e9d8cc833fe596b5c. Latest consensus state: current_term: 1 leader_uuid: "f22573f41dfc448e9d8cc833fe596b5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.558007 28898 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 18:56:07.575088 28863 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f22573f41dfc448e9d8cc833fe596b5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f22573f41dfc448e9d8cc833fe596b5c" member_type: VOTER last_known_addr { host: "127.23.218.254" port: 36105 } } peers { permanent_uuid: "ddb7a0fe16d942479a476f0add03a597" member_type: VOTER last_known_addr { host: "127.23.218.253" port: 34085 } } peers { permanent_uuid: "7d8dea1554dc4519b61eb5a43ff3d558" member_type: VOTER last_known_addr { host: "127.23.218.252" port: 38907 } } }
I20260809 18:56:07.575178 28863 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c [sys.catalog]: This master's current role is: LEADER
I20260809 18:56:07.577205 28878 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 18:56:07.577899 28878 catalog_manager.cc:1504] Loading token signing keys...
I20260809 18:56:07.587659 28878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f22573f41dfc448e9d8cc833fe596b5c: Generated new TSK 0
I20260809 18:56:07.587817 28878 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 18:56:07.620101 28564 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.618669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47176 (local address 127.23.218.254:36105)
0809 18:56:07.618711 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.618713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.619493 (+ 780us) server_negotiation.cc:407] Connection header received
0809 18:56:07.619588 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.619589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.619600 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.619628 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.620039 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47176: BlockingRecv error: recv got EOF from 127.0.0.1:47176 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 18:56:07.620786 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.618980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.23.218.253:34085)
0809 18:56:07.619014 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.619015 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.620321 (+ 1306us) server_negotiation.cc:407] Connection header received
0809 18:56:07.620409 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.620410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.620420 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.620447 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.620731 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35836: BlockingRecv error: recv got EOF from 127.0.0.1:35836 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 18:56:07.621479 28803 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.619253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.23.218.252:38907)
0809 18:56:07.619290 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.619292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.620993 (+ 1701us) server_negotiation.cc:407] Connection header received
0809 18:56:07.621094 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.621095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.621104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.621132 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.621424 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57592: BlockingRecv error: recv got EOF from 127.0.0.1:57592 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 18:56:07.625993 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.622558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35844 (local address 127.23.218.253:34085)
0809 18:56:07.622589 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.622591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.625525 (+ 2934us) server_negotiation.cc:407] Connection header received
0809 18:56:07.625615 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.625616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.625632 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.625656 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.625950 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35844: BlockingRecv error: recv got EOF from 127.0.0.1:35844 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 18:56:07.628736 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.628035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.23.218.253:34085)
0809 18:56:07.628065 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.628067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.628304 (+ 237us) server_negotiation.cc:407] Connection header received
0809 18:56:07.628388 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.628389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.628398 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.628423 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.628700 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35860: BlockingRecv error: recv got EOF from 127.0.0.1:35860 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 18:56:07.632473 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.631754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.23.218.253:34085)
0809 18:56:07.631784 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.631786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.631996 (+ 210us) server_negotiation.cc:407] Connection header received
0809 18:56:07.632088 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.632089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.632100 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.632127 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.632436 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35866: BlockingRecv error: recv got EOF from 127.0.0.1:35866 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 18:56:07.638195 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.637471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35876 (local address 127.23.218.253:34085)
0809 18:56:07.637503 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.637505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.637733 (+ 228us) server_negotiation.cc:407] Connection header received
0809 18:56:07.637826 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.637826 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.637836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.637863 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.638157 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35876: BlockingRecv error: recv got EOF from 127.0.0.1:35876 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 18:56:07.647893 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.647137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35888 (local address 127.23.218.253:34085)
0809 18:56:07.647169 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.647170 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.647423 (+ 253us) server_negotiation.cc:407] Connection header received
0809 18:56:07.647516 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.647517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.647526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.647554 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.647855 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35888: BlockingRecv error: recv got EOF from 127.0.0.1:35888 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 18:56:07.665504 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.664897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35896 (local address 127.23.218.253:34085)
0809 18:56:07.664929 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.664931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.665064 (+ 133us) server_negotiation.cc:407] Connection header received
0809 18:56:07.665094 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.665095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.665105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.665170 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.665462 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35896: BlockingRecv error: recv got EOF from 127.0.0.1:35896 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 18:56:07.698879 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.698308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.23.218.253:34085)
0809 18:56:07.698374 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.698376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.698448 (+ 72us) server_negotiation.cc:407] Connection header received
0809 18:56:07.698487 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.698488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.698498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.698559 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.698828 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35908: BlockingRecv error: recv got EOF from 127.0.0.1:35908 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 18:56:07.764285 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.763784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.23.218.253:34085)
0809 18:56:07.763838 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.763840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.763904 (+ 64us) server_negotiation.cc:407] Connection header received
0809 18:56:07.763947 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.763948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.763957 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.764010 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.764248 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35920: BlockingRecv error: recv got EOF from 127.0.0.1:35920 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 18:56:07.894017 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:07.893370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35936 (local address 127.23.218.253:34085)
0809 18:56:07.893427 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:07.893429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:07.893529 (+ 100us) server_negotiation.cc:407] Connection header received
0809 18:56:07.893596 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:07.893597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:07.893608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:07.893654 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:07.893967 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35936: BlockingRecv error: recv got EOF from 127.0.0.1:35936 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 18:56:08.151628 28695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 18:56:08.151123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.23.218.253:34085)
0809 18:56:08.151177 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:08.151179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:08.151190 (+ 11us) server_negotiation.cc:407] Connection header received
0809 18:56:08.151223 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:08.151224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:08.151234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:08.151281 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 18:56:08.151568 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35946: BlockingRecv error: recv got EOF from 127.0.0.1:35946 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 18:56:08.487787 28832 catalog_manager.cc:1259] Loaded cluster ID: ad0c19783401474ab515c13f777f96ce
I20260809 18:56:08.487845 28832 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558: loading cluster ID for follower catalog manager: success
I20260809 18:56:08.488610 28832 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558: acquiring CA information for follower catalog manager: success
I20260809 18:56:08.488881 28832 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d8dea1554dc4519b61eb5a43ff3d558: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 18:56:08.557086 28914 catalog_manager.cc:1259] Loaded cluster ID: ad0c19783401474ab515c13f777f96ce
I20260809 18:56:08.557142 28914 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597: loading cluster ID for follower catalog manager: success
I20260809 18:56:08.557863 28914 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597: acquiring CA information for follower catalog manager: success
I20260809 18:56:08.558113 28914 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ddb7a0fe16d942479a476f0add03a597: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 18:56:08.666864 29032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:08.664599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.23.218.253:34085)
0809 18:56:08.664729 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:08.664731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:08.664741 (+ 10us) server_negotiation.cc:407] Connection header received
0809 18:56:08.664806 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:08.664806 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:08.664855 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:08.664923 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:08.665043 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.665538 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.666227 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.666427 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.666469 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:08.666589 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:08.666697 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:08.666751 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:08.666763 (+ 12us) server_negotiation.cc:299] Negotiation successful
0809 18:56:08.666781 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260809 18:56:08.670410 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:08.668453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57596 (local address 127.23.218.252:38907)
0809 18:56:08.668560 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:08.668562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:08.668571 (+ 9us) server_negotiation.cc:407] Connection header received
0809 18:56:08.668608 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:08.668608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:08.668643 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:08.668710 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:08.668787 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.669240 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.669854 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.670028 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.670066 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:08.670162 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:08.670243 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:08.670292 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:08.670320 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 18:56:08.670335 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
I20260809 18:56:08.674415 29032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:08.671865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.23.218.253:34085)
0809 18:56:08.671912 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:08.671913 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:08.672089 (+ 176us) server_negotiation.cc:407] Connection header received
0809 18:56:08.672115 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:08.672115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:08.672126 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:08.672170 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:08.672310 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.672767 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.673701 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.673940 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.673994 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:08.674148 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:08.674179 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:08.674220 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:08.674307 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 18:56:08.674329 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 18:56:08.674779 29052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:08.671748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.23.218.254:36105)
0809 18:56:08.671843 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:08.671845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:08.671853 (+ 8us) server_negotiation.cc:407] Connection header received
0809 18:56:08.671893 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:08.671894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:08.671942 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:08.672164 (+ 222us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:08.672207 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.672976 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.673943 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.674228 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.674293 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:08.674480 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:08.674576 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:08.674637 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:08.674688 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 18:56:08.674708 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":44,"threads_started":1}
I20260809 18:56:08.674827 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 18:56:08.672112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57606 (local address 127.23.218.252:38907)
0809 18:56:08.672154 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 18:56:08.672156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 18:56:08.672163 (+ 7us) server_negotiation.cc:407] Connection header received
0809 18:56:08.672183 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 18:56:08.672184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 18:56:08.672192 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 18:56:08.672234 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 18:56:08.672348 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.673043 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.673742 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 18:56:08.674016 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 18:56:08.674073 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 18:56:08.674207 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 18:56:08.674234 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 18:56:08.674286 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 18:56:08.674661 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 18:56:08.674683 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 18:56:08.676045 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 28445
I20260809 18:56:08.688977 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 28573
I20260809 18:56:08.695709 24427 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUXKsiJ/build/release/bin/kudu with pid 28699
2026-08-09T18:56:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2895 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5766 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (269163 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST